Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] Some practical hwloc API feedback
From: Ralph Castain (rhc_at_[hidden])
Date: 2011-09-23 12:04:24


On Sep 22, 2011, at 3:05 PM, Brice Goglin wrote:

> Le 22/09/2011 22:42, Ralph Castain a écrit :
>> I guess I didn't get that from your documentation. Since caches sit
>> between socket and core, they appear to affect the depth of the core
>> in a given socket. Thus, if there are different numbers of caches in
>> the different sockets on a node, then the core/pu level would change
>> across the sockets.
>
> No, the level always contain all elements of the same type (+depth for
> caches), even if they are not at the same "distance" to the root (not
> "depth").
>
> Let's say you have two single-core sockets. One with no cache. One with
> a L1.
> What happens is:
> * first level/depth is socket, contains two sockets, cover all cores.
> * level 2 is L2, single element, *does not cover all cores*
> * level 3 is core, two elements.
>
> The funky thing here is that the parent/child links between the first
> socket and its core go across level 2 because nothing matches there. In
> the first socket, you have Socket(depth1)->Core(depth3) while in the
> second socket you have Socket(depth1)->Cache(depth2)->Core(depth3)
>
> So what we call "depth" in hwloc, is not the number of parent/child
> links between you and the root, it's really the number of levels between
> you and the root, even if you don't have any parent in some of these levels.
>
> Looks like we need to clarify this :)
>

Indeed - having the above example in hwloc.h would help. I think the key thing here is that the depth for a given type is being set across the entire node, and not by the local structure - i.e., the depth of the core in your example is determined by looking across the node at the max depth of any core in its local structure. Those of us coming from the chip world will find that confusing, as we look at things one socket at a time, but we can adapt.

All that said, if I put my dictionary away and can get the code to work, hopefully we won't have to parse thru it again. :-)

Thanks!