Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] hwloc on power7
From: Jirka Hladky (jhladky_at_[hidden])
Date: 2010-08-05 09:17:07


Hi Brice,

I have discussed the issue internally and we can confirm that CPU topology for
POWER7 as reported by 2.6.32 kernel (in RHEL6.0) is indeed wrong.

I have submitted BZ against RHEL6.0
https://bugzilla.redhat.com/show_bug.cgi?id=621546

to get it fixed. Let's see if the fix can be back-ported from 2.6.34 kernel.

Thanks
Jirka

On Thursday, August 05, 2010 07:08:09 am Brice Goglin wrote:
> Power7 topology isn't properly reported by old kernels. We've been said
> that it works fine with 2.6.34. I am not sure which commit fixed this. I
> don't see many commits talk about Power7 topology between 2.6.32 and
> 2.6.34, so it may be this one (from 2.6.34):
>
> commit 4b83c330b4d38e869111bda6e9077d4f61ed974a
> Author: Anton Blanchard <anton_at_[hidden]>
> Date: Wed Apr 7 15:33:44 2010 +0000
>
> powerpc/numa: Add form 1 NUMA affinity
>
> Firmware changed the way it represents memory and cpu affinity on
> POWER7. Unfortunately the old method now caps the topology to work around
> issues with legacy operating systems. For Linux to get the correct
> topology we need to use the new form 1 affinity information.
>
> We set the form 1 field in the client architecture, and if we see "1"
> in the ibm,associativity-form property firmware supports form 1 affinity
> and we should look at the first field in the
> ibm,associativity-reference-points array. If not we use the second field
> as we always have.
>
> Signed-off-by: Anton Blanchard <anton_at_[hidden]>
> Signed-off-by: Benjamin Herrenschmidt <benh_at_[hidden]>
>
>
> Maybe some distros should backport it...
>
> Brice
>
> Le 05/08/2010 00:19, Jirka Hladky a écrit :
> > Hi,
> >
> > I just got access to one POWER7 box. Both hwloc 1.0.2 and latest hwloc
> > 1.1 seem to have problems to detect the sockets correctly.
> >
> > It's IBM 8406-71Y server equipped with only one socket. On the chip there
> > are 8 cores. lstopo seems to be wrong here, reporting 8 sockets.
> >
> > I'm not sure if this problem arose in linux kernel or it's bug in lstopo
> > code. Does anybody else tried to run hwloc on POWER7?
> >
> > Thanks
> > Jirka
> >
> >
> > Kernel: 2.6.32-54.el6.ppc64
> >
> > root_at_[hidden]: /tmp/hwloc-1.1a1r2387/utils
> > $ ./lstopo
> > Machine (7616MB)
> >
> > Socket #0 + L3 #0 (4096KB) + L2 #0 (256KB) + L1 #0 (32KB) + Core #0
> >
> > PU #0 (phys=0)
> > PU #1 (phys=1)
> > PU #2 (phys=2)
> > PU #3 (phys=3)
> >
> > Socket #1 + L3 #1 (4096KB) + L2 #1 (256KB) + L1 #1 (32KB) + Core #1
> >
> > PU #4 (phys=4)
> > PU #5 (phys=5)
> > PU #6 (phys=6)
> > PU #7 (phys=7)
> >
> > Socket #2 + L3 #2 (4096KB) + L2 #2 (256KB) + L1 #2 (32KB) + Core #2
> >
> > PU #8 (phys=8)
> > PU #9 (phys=9)
> > PU #10 (phys=10)
> > PU #11 (phys=11)
> >
> > Socket #3 + L3 #3 (4096KB) + L2 #3 (256KB) + L1 #3 (32KB) + Core #3
> >
> > PU #12 (phys=12)
> > PU #13 (phys=13)
> > PU #14 (phys=14)
> > PU #15 (phys=15)
> >
> > Socket #4 + L3 #4 (4096KB) + L2 #4 (256KB) + L1 #4 (32KB) + Core #4
> >
> > PU #16 (phys=16)
> > PU #17 (phys=17)
> > PU #18 (phys=18)
> > PU #19 (phys=19)
> >
> > Socket #5 + L3 #5 (4096KB) + L2 #5 (256KB) + L1 #5 (32KB) + Core #5
> >
> > PU #20 (phys=20)
> > PU #21 (phys=21)
> > PU #22 (phys=22)
> > PU #23 (phys=23)
> >
> > Socket #6 + L3 #6 (4096KB) + L2 #6 (256KB) + L1 #6 (32KB) + Core #6
> >
> > PU #24 (phys=24)
> > PU #25 (phys=25)
> > PU #26 (phys=26)
> > PU #27 (phys=27)
> >
> > Socket #7 + L3 #7 (4096KB) + L2 #7 (256KB) + L1 #7 (32KB) + Core #7
> >
> > PU #28 (phys=28)
> > PU #29 (phys=29)
> > PU #30 (phys=30)
> > PU #31 (phys=31)
> >
> > _______________________________________________
> > hwloc-devel mailing list
> > hwloc-devel_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel
>
> _______________________________________________
> hwloc-devel mailing list
> hwloc-devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel