Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] hwloc-bind syntax
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2009-12-04 05:36:25


Jeff Squyres wrote:
> It might be good to safely ignore 0x if it's present, but that's a small feature enhancement that can be done at any time (I filed a future ticket).
>

It seems to work actually :)

>> We might want to drop the Linux "cpuset" word and use "cgroup" instead.
>> Both are supported by Linux, but the latter now contains the former and
>> more, so people are supposed to use cgroup now. hwloc supports both.
>>
>
> Linux is likely to be among the most popular target for hwloc -- so can you explain in good words definitions for the following:
>
> - hwloc cpuset
>

Opaque structure describing a set of logical processors. Each hwloc
object structure contains a cpuset field that describes which logical
processors are contained in the corresponding physical object. hwloc
cpusets are used by hwloc binding routines.

> - Linux cpuset
> - Linux cgroup
>

See http://www.mjmwired.net/kernel/Documentation/cgroups.txt, and look
for cpusets in there:

Control Groups provide a mechanism for aggregating/partitioning sets of
tasks, and all their future children, into hierarchical groups with
specialized behaviour.
[...]
On their own, the only use for cgroups is for simple job tracking.
The intention is that other subsystems hook into the generic
cgroup support to provide new attributes for cgroups, such as
accounting/limiting the resources which processes in a cgroup can
access. For example, cpusets allows you to associate a set of CPUs and
a set of memory nodes with the tasks in each cgroup.

> Additionally -- the word "father" is used in the docs. Should we use the gender-neutral "parent" instead?
>

I am not sure. The object structure contains a father pointer. We use
parent in the API, but it might refer to different things, like father,
grandfather, ...

>> You don't care about starting with system or something else. You can
>> ignore the system level as you could ignore the socket level between
>> nodes and cores.
>>
>> If you have 1 system with 2 nodes with 2 sockets each with 2 cores each,
>> you get:
>> node:1 core:2 is equivalent to system:0 node:1 socket:2 core:0 and
>> equivalent to system:0 core:6
>>
>
> Did you mean:
>
> node:1.core:2 == system:0.node:1.socket:2.core:0 == system:0.core:6
>
> ?
>

Yes.

> What I meant by my question was -- aren't the 3 diagrams above equivalent to "core:6"? If so, what's the value of the foo.bar.baz notation?

If you have a 96 core machine like we do, the hierarchical notation
(foo.bar.baz) is really nice. If I want to bind on
node:2.socket:3.core:4, it's much easier than looking at the topology
and finding that it's core:70. Using physical or logical indexes doesn't
change anything here. I agree that we don't do that often in real
applications, but I actually use that quite a lot for my own debugging :)

I actually don't see why people would like to use physical numbers in
such a hierarchical notation since physical socket/core numbers are
often strange/illogical and nobody remembers them. However, I agree that
the physical indexes are useful when *not* using a hierarchical
notation, ie I want to bind on thread OS index #46.

Brice