Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] towards PLPA-like API in 1.0
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2009-11-05 11:22:15


Hello,

I've been looking at the PLPA API and here's what we could do to add a
hwloc/plpa.h offering kind of the same features.

* PLPA-like API is prefixed with hwloc_plpa_ and all functions get a new
hwloc_topology_t parameter. The problematic ones are:

+ int hwloc_plpa_sched_getaffinity(pid_t pid, hwloc_cpuset_t cpuset);

It's just a hwloc_get_cpubind(), but we don't have it since it would not
be supported on all OS. But I think we should add it anyway.

+ int hwloc_plpa_get_core_flags(hwloc_topology_t topology, int socket_id, int core_id, int *exists, int *online);

Is says whether a core (given by core+socket os_index) exists and is
online. First, we don't have topology information about offline
processors. Secondly, on Nehalem you can disable a single thread within
a hyperthreaded core, so an "offline core" doesn't mean much. I would
just vote for returning whether the core exists and remove the online
return value here (see below for more about offline CPUs).

+ int hwloc_plpa_map_to_processor_id(hwloc_topology_t topology, int socket_id, int core_id, int *processor_id);

Return the os_index of the processor given by socket/core os_index. If
we have hyperthreading, all thread sibling are valid here. I would just
return the os_index of first one since this function was supposed to be
used for binding anyway.

* Then we have all count-spec related API, which lets you look for
information about all processors, or all online ones, or all offline ones.

If people are really interested with offline CPUs, they can look at the
get_offline_cpuset below. There is no topology information about offline
CPUs on Linux anyway, so I am not sure it's worth trying to manage
offline and online CPUs in a uniform way. I would rather remove the
count-spec argument and just only work on available/online/enabled
processors with:

+ int hwloc_plpa_get_processor_data(hwloc_topology_t topology, int *num_processors, int *max_processor_id);

+ int hwloc_plpa_get_processor_id(int processor_num, int *processor_id);

* Probing

>From what I understand, plpa_have_topology_information() tells whether
PLPA knows what's in the hardware, while plpa_api_probe() tells whether
binding is supported. We could add:

+ hwloc_topology_support(hwloc_topology_t topology, unsigned *support)

which fills "support" with a bitmask of things like OS is supported,
binding a thread is possible, binding a processor is possible, getting
the binding of a process is possible, ...

Then we could reimplement

+ int hwloc_plpa_have_topology_information(hwloc_topology_t topology);
+ int hwloc_plpa_api_probe(hwloc_topology_t topology);

* New hwloc general-purpose helpers that will be used by PLPA wrappers
but may still be useful for other reasons:

+ hwloc_obj_t hwloc_get_parent_obj_by_type(hwloc_topology_t topology, hwloc_obj_type_t type, hwloc_obj_t obj);
+ hwloc_obj_t hwloc_get_parent_obj_by_depth(hwloc_topology_t topology, unsigned depth, hwloc_obj_t obj);

Walks the parent chain until it finds the right type (or depth).

+ hwloc_obj_t hwloc_get_proc_obj_by_os_index(hwloc_topology_t topology, unsigned proc_os_index);

Given a processor os_index, returns the corresponding object.

+ hwloc_cpuset_t hwloc_topology_get_offline_cpuset(hwloc_topology_t topology);

Returns a CPU set of existing CPUs that are offline, disabled by
administrator, or unavailable to this process if we're restricting the
topology to the process origin binding for instance. I am not sure we
actually need to distinguish all these cases.

* Finally, I plan to reimplement the PLPA tools, either in tests/plpa/
or as a real (installed) tools for a transition period.

+ plpa-info already works in my tree. Are there people that really need
it? "lstopo -v -" basically shows the same and even more (offline CPUs
are not reported in the trunk but I modified my tree to print the number
of offline CPUs and the corresponding cpuset).

+ plpa-taskset needs a lot of work for convering its own cpuset stuff
into ours. It has an advanced binding syntax that some people may be
used to. hwloc-bind has an advanced but different syntax. Apart from
that, the features are the same.

By the way, I wonder if we want to add public functions converting
between cpusets (0x0f00ffff) and cpulist string (0-15,24-27)
(plpa-taskset uses something like this).

Brice