Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] [hwloc-svn] svn:hwloc r2257
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2010-09-17 03:53:29


Le 29/06/2010 11:26, Brice Goglin a écrit :
> Le 29/06/2010 11:15, bgoglin_at_[hidden] a écrit :
>
>> Author: bgoglin
>> Date: 2010-06-29 05:15:57 EDT (Tue, 29 Jun 2010)
>> New Revision: 2257
>> URL: https://svn.open-mpi.org/trac/hwloc/changeset/2257
>>
>> Log:
>> Add hwloc_get_distances() which returns a matrix of distances
>> for the given type.
>>
>>
> Some notes about all this:
> * The matrix that I keep inside the topology is ordered by os_indexes
> since logical indexes may change later in the discovery (depending on
> the matrix itself, when we group according to distances)
> * I am recomputing the logically-ordered matrix on demand, but we could
> also recompute it once at the end of the discovery
> * I wonder if the current code works if the NUMA physical indexes are
> non-contigous
>

These are fixed now, the code looks much better. The main topology only
contains a logically-ordered matrix that is computed at the end of the
discovery. Linux keeps its custom nonsparse-physically-ordered matrix in
its backend-specific params and converts it at the end of the discovery
once logical indexes are ready.

Brice

> * The matrix is manipulated as a mix of unsigned[nbobjs*nbobjs] and
> unsigned[nbobjs][nbobjs]. I don't know the rules well enough to be sure it's
> safe :)
> * I only handle NUMA distances so far. No other type is supported while
> the grouping code also computes the distance between groups of NUMA
> nodes (or groups of groups). It's difficult to store matrices for random
> types of objects since there could be multiple matrices for a single
> type (different levels of NUMA groups, ...) and the depth of these might
> vary later in the discovery (if we insert another level of group above
> the current one, or some misc objects?). A good solution might be to
> attach matrices to the topology using an identifier such as Type +
> Type-specific depth (NUMANode, or Group#0, Group#1, L2Cache, ...). Needs
> some work... Having such a unique level identifier might be good for
> other reasons.
>