Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] Bug report: topology strange on SGI UltraViolet
From: Brice Goglin (brice.goglin_at_[hidden])
Date: 2010-07-29 07:01:10

> To my opinion, the job hwloc does in forming "groups" is basically OK.
> Also the group content makes sense.

We're lucky that it somehow matches the physical ordering,
but it is really meaningless given the distance matrix.
That's why Group2 matches nothing in reality.
Group3 matches nothing as well from what I understand.

This meaningless part will disappear in hwloc 1.1. You will
only see 24 Group0 objects.

> The only "strange" thing is, that the grouping code becomes disturbed on
> this special machine, which only contains 3/4 of the NUMA nodes that are
> found in a fully equipped rack.

It's an artifact of the aforementioned meaningless grouping code.
If you have 2^N objects with such a distance matrix, the grouping code
will create a binary tree. If it's not 2^N, you'll artifact like here
since the binary tree isn't complete.

> Physically, the 2nd enclosure is only
> half filled. I'm wondering what would happen in a fully equipped rack.
> Will there be 4xGroup3, or 2xGroup4 with 2xGroup3 each? From my feeling
> the latter should be happen.

Yes, the latter would happen. Such a distance matrix always groups pairs
of consecutive objects starting from #0. So you'll get two pairs of Group3
grouped in 2 Group4 objects.