Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] Strange difference
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2010-03-30 05:58:55


Chris Samuel wrote:
> On Sat, 27 Mar 2010 05:15:50 am Jeff Squyres wrote:
>
>
>> At least on my machine, the output width is still far less than 80
>> characters, so the full word should be no problem. But I don't know if
>> there are other strange topologies out there that would take up more
>> space...?
>>
>
> Be interesting to see what it would look like on a large SGI UV, for
> instance..
>

On a old Altix 4700 with 256 itanium cores, it's still far from 80 columns:

Machine (493GB)
  Misc1 #0 (123GB)
    Misc0 #0 (31GB)
      NUMANode #0 (phys=0 7875MB)
        Socket #0
          Core #0 + P #0 (phys=0)
          Core #1 + P #1 (phys=1)
        Socket #1
          Core #2 + P #2 (phys=2)
          [...]

And in verbose mode:

Machine (phys=0 total=516912176KB)
  Misc1 #0 (total=129224048KB)
    Misc0 #0 (total=32296336KB)
      NUMANode #0 (phys=0 local=8064400KB total=8064400KB)
        Socket #0 (phys=0)
          Core #0 (phys=0)
            P #0 (phys=0)
          Core #1 (phys=1)
            P #1 (phys=1)

(generated from sysfs tarballs, but shouldn't be different from the
actual machine)

On a small Altix UV with 24 Nehalem-EX cores with HT, the width is similar.

IIRC, the largest UV is 2048 cores, the hierarchy should be
* 1 machine
* 256 NUMA nodes grouped in 3 or 4 levels of "misc" objects depending on
their distances
* in each NUMA Node, 1 socket x 1 L3 x 8 L2 x 1 L1 x 1 core x 2 threads
That's 12 levels of hierarchy, which means 22 characters of indentation
in the lstopo output. Each line is usually 10-20 characters, except when
you have some memory (it could be 40-50 characters in this case). So we
should be ok.

Brice