Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: [hwloc-devel] PUs not located under cores
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-06-24 12:02:48


I have another problem report for OMPI hwloc support from IBM -- they have a PPC-based machine with output that looks like this:

-----
[root_at_p6ihhpc5 tests]# hwloc-info
depth 0: 1 Machine (type #1)
 depth 1: 4 NUMANodes (type #2)
  depth 2: 64 PUs (type #6)
[root_at_p6ihhpc5 tests]# hwloc-ls
Machine (123GB)
  NUMANode #0 (phys=0 30GB)
    PU #0 (phys=0)
    PU #1 (phys=1)
    PU #2 (phys=2)
    PU #3 (phys=3)
    PU #4 (phys=4)
    PU #5 (phys=5)
    PU #6 (phys=6)
    PU #7 (phys=7)
    PU #8 (phys=8)
    PU #9 (phys=9)
    PU #10 (phys=10)
    PU #11 (phys=11)
    PU #12 (phys=12)
    PU #13 (phys=13)
    PU #14 (phys=14)
    PU #15 (phys=15)
  NUMANode #1 (phys=1 31GB)
    PU #16 (phys=16)
    PU #17 (phys=17)
    PU #18 (phys=18)
    PU #19 (phys=19)
    PU #20 (phys=20)
    PU #21 (phys=21)
    PU #22 (phys=22)
    PU #23 (phys=23)
    PU #24 (phys=24)
    PU #25 (phys=25)
    PU #26 (phys=26)
    PU #27 (phys=27)
    PU #28 (phys=28)
    PU #29 (phys=29)
    PU #30 (phys=30)
    PU #31 (phys=31)
  NUMANode #2 (phys=2 31GB)
    PU #32 (phys=32)
    PU #33 (phys=33)
    PU #34 (phys=34)
    PU #35 (phys=35)
    PU #36 (phys=36)
    PU #37 (phys=37)
    PU #38 (phys=38)
    PU #39 (phys=39)
    PU #40 (phys=40)
    PU #41 (phys=41)
    PU #42 (phys=42)
    PU #43 (phys=43)
    PU #44 (phys=44)
    PU #45 (phys=45)
    PU #46 (phys=46)
    PU #47 (phys=47)
  NUMANode #3 (phys=3 31GB)
    PU #48 (phys=48)
    PU #49 (phys=49)
    PU #50 (phys=50)
    PU #51 (phys=51)
    PU #52 (phys=52)
    PU #53 (phys=53)
    PU #54 (phys=54)
    PU #55 (phys=55)
    PU #56 (phys=56)
    PU #57 (phys=57)
    PU #58 (phys=58)
    PU #59 (phys=59)
    PU #60 (phys=60)
    PU #61 (phys=61)
    PU #62 (phys=62)
    PU #63 (phys=63)
-----

Is that right? I.e., is it proper that hwloc can return just a pile of PUs under a numa node with no core as an umbrella?

I can certainly code around this if we expect this kind of output from hwloc -- the current code using hwloc topology info in OMPI assumes that PUs will be under cores. But before I did that, I wanted to ask if this was expected.

It might be worth adding a section in hwloc.doxy describing stuff like this (yes, I'm volunteering to write it :-) ) -- that using hwloc can still result in lots of different kinds of outputs, depending on the type of machine you're running on. Case in point is the above output vs. the output from a typical intel/amd-based machine where you have a topology containing a machine, sockets, various caches, numa nodes, cores, and PU's.

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/