Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] BGQ question.
From: Christopher Samuel (samuel_at_[hidden])
Date: 2014-03-30 20:25:02


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi John,

On 28/03/14 21:04, Biddiscombe, John A. wrote:

> Just as a follow up to this thread. I spoke with someone from IBM
> and they tell me that 2 cores of 4 hardware threads each are hidden
> from the kernel (how do they do that?) and used for the custom HS4
> cards we have installed on the IO nodes, which explains why I see
> only 60 instead of 68 threads.

Interesting, I suspected they were masked off for some reason. What
does the following say on one of these?

cat /sys/devices/system/cpu/{kernel_max,present,offline,present}

> the 2 bgvrnic tasks I see spinning at 100% run on threads 58/59 and
> service the connection from ION to CN.

Yup, those are the processes that are faking the bgvrnic_0 IB
interface that is really presenting the 5D torus.

> It looks as though everything is reporting as expected - as long as
> I compile hwloc on the ION itself, it seems to be correct.

Great!

> Thanks and sorry for any misunderstanding

No need to apologise, we've learnt something about the BGAS kernel
patches.

All the best,
Chris
- --
 Christopher Samuel Senior Systems Administrator
 VLSCI - Victorian Life Sciences Computation Initiative
 Email: samuel_at_[hidden] Phone: +61 (0)3 903 55545
 http://www.vlsci.org.au/ http://twitter.com/vlsci

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.14 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/

iEYEARECAAYFAlM4td4ACgkQO2KABBYQAh8iRwCeLsPkiKsFTqGEJzVs4fGsg8nc
NskAniCSwL2wMiR+ovIkikZolaAPyZ7X
=gxFB
-----END PGP SIGNATURE-----