Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] Using distances
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-04-21 07:09:41


On 21/04/2012 12:23, Jeffrey Squyres wrote:
> I'm trying to use hwloc distances in Open MPI (e.g., find the distance from each OpenFabrics device to the PU(s) where this process is bound), and I'm a bit confused by the distances documentation.
>
> If I have a WHOLE_SYSTEM topology, and I know that this process is bound to one or more PUs (e.g., both PUs in a core), can you summarize how I use the hwloc distances functionality to determine the distance from my process to each of the OF devices?
>

I assume you have the entire distance (latency) matrix between all NUMA
nodes as usually reported by the BIOS.

const struct hwloc_distance_s *distances =
hwloc_get_whole_distance_matrix_by_type(topology, HWLOC_OBJ_NODE);
assert(distances);
assert(distances->latency);

Now distances->latency[a+b*distances->nbobjs] contains the latency
between NUMA nodes whose *logical* indexes are a and b (it may be
asymmetrical).

Now get the NUMA node object close to your PUs and the NUMA objects
close to each OFED device, take their ->logical_index and you'll get the
latencies.

Brice