Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] BGQ empty topology with MPI
From: Christopher Samuel (samuel_at_[hidden])
Date: 2012-03-22 22:41:02

Hash: SHA1

On 22/03/12 01:08, Daniel Ibanez wrote:

> Attached is the stderr and stdout from lstopo compiled as you
> said.

Interesting, so it's not correctly detecting the topology as BG/Q is
16 compute cores, each with 4 hardware threads. Instead it's
detecting all 64 hardware threads and treating them as cores if I'm
reading that

I was puzzled by the OS info output too, it says:

Machine#0(Backend=Linux OSName=CNK
OSRelease=2.6.32-220.el6.bgq110_20120104.ppc64 OSVersion=1
HostName=R00-ID-J04.i2b.cetus Architecture=) cpuset 0xf...f complete
0xffffffff,0xffffffff online 0xf...f allowed 0xf...f nodeset 0x0
completeN 0x0 allowedN 0xf...f

However, looking at the (open) source code for the CNK [1] (at least
for BG/P) the uname info seems to be derived from the I/O nodes when
its running in CIOD mode, so I suspect that's what's happening here
(looks like a RHEL6 derived kernel from that).

> I can't run on the compute nodes since its
> a script, but I can run it on the front end node.

For those unfamiliar with BlueGene (at least P, and I suspect the same
is true for Q), this is because the CNK doesn't implement fork() or
execve(), they're designed to start your code and just keep running it
until it dies.

[1] -

- --
    Christopher Samuel - Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel_at_[hidden] Phone: +61 (0)3 903 55545

Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla -