Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] CPU affinity of OS Devices?
From: Guy Streeter (streeter_at_[hidden])
Date: 2012-11-07 10:56:31

On 11/06/2012 05:20 PM, Brice Goglin wrote:
> Le 06/11/2012 23:55, Guy Streeter a écrit :
>> On 11/06/2012 03:53 PM, Brice Goglin wrote:
>>> Hello Guy,
>>> I don't think OS devices ever had a cpuset. All objects that are not
>>> things where you can bind processes usually have NULL cpusets. So when
>>> you have a PCI or OS device, you walk up the obj->parent pointer until
>>> you find an object with a non-NULL cpuset. That's the affinity you're
>>> looking for.
>>> You can use hwloc_get_non_io_ancestor_obj() (in hwloc/helper.h) to find
>>> the first parent with non-NULL cpuset.
>>> Brice
>> I didn't mean to imply that they had gone away. My question is how do I
>> specify a binding like "not on the same CPU that is handling the Ethernet
>> interrupts"?
> Assuming you have the OS device for this interrupt, find the parent
> object whose cpuset field isn't NULL, reverse this cpuset with
> hwloc_bitmap_not() and bind to some object that is inside the resulting
> cpuset.
> Brice
The problem with that is that it will exclude all of the CPUs associated with
the node, or all of the CPUs on the machine if it isn't NUMA. The CPU affinity
of the IRQ can be set independent of the bounding object of the PCI bridge the
device is below.
On my machine, the ahci device uses IRQ 90, and the affinity of that IRQ is
CPU 3. The first parent with a cpuset above the OSDev block object for my disk
drive is the machine itself.