Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] hwloc to be included in RHEL 6.1
From: Jirka Hladky (jhladky_at_[hidden])
Date: 2010-11-18 09:50:06


Hi all,

Red Hat would like to included hwloc in the upcoming version of the Red Hat
Enterprise Linux 6.1. There is Bugzilla 648593
[RFE] Include Portable Hardware Locality (hwloc) in RHEL

https://bugzilla.redhat.com/show_bug.cgi?id=648593

to address this.

I got following input from the devel:
=================================================
There appears to be a significant drawback to using hwloc. The core # shown in
hwloc-ls does NOT map 1:1 with the processor id in /proc/cpuinfo.

For example, on intel-s3e36-02.lab hwloc shows the core ids in socket 0 as
{0,1,2,3,4,5,6,7}.

/proc/cpuinfo shows these as physically being {0,4,8,12,16,20,24,28}.

On the cmd-line, hwloc-ls does indicate a difference between the hwloc core id
and the physical id:

[root_at_intel-s3e36-02 ~]# hwloc-ls
Machine (64GB)
  NUMANode #0 (phys=0 16GB) + Socket #0 + L3 #0 (24MB)
    L2 #0 (256KB) + L1 #0 (32KB) + Core #0 + PU #0 (phys=0)
    L2 #1 (256KB) + L1 #1 (32KB) + Core #1 + PU #1 (phys=4)
    L2 #2 (256KB) + L1 #2 (32KB) + Core #2 + PU #2 (phys=8)
    L2 #3 (256KB) + L1 #3 (32KB) + Core #3 + PU #3 (phys=12)
    L2 #4 (256KB) + L1 #4 (32KB) + Core #4 + PU #4 (phys=16)
    L2 #5 (256KB) + L1 #5 (32KB) + Core #5 + PU #5 (phys=20)
    L2 #6 (256KB) + L1 #6 (32KB) + Core #6 + PU #6 (phys=24)
    L2 #7 (256KB) + L1 #7 (32KB) + Core #7 + PU #7 (phys=28)

If you use the graphical interface, it is possible that customers/GSS/everyone
screws up the reporting of CPU #s.

Possible solution: Have hwloc-ls use '-p' by default.
=================================================

I'm not sure if you are open to change the default from --logical to --
physical. Please let me know your opinion on it. If you don't think that it's
a good idea perhaps you can give us arguments why you prefer logical indexing
over physical indexing.

Another point is that at the moment you cannot distinguish if the graphical
output (.png, X, ...) was created with lstopo --physical or lstopo --logical.

Could you please add the legend to the picture explaining which index was
used?

Example:
===================================================
This picture is using logical indexing. Logical indexes are artificial (hwloc
internal) indexes. They start from 0 and incremented by 1 for each object.

This picture is using physical indexing. Indexes map 1:1 to indexes used by
kernel as in /proc/cpuinfo or shown by top utility.
====================================================

Please let me know your opinion on this matter.

Thanks a lot!
Jirka