Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] Running OpenMPI on SGI Altix with 4096 cores : very poor performance
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2011-01-07 10:32:49


Well, bummer -- there goes my theory. According to the hwloc info you posted earlier, this shows that OMPI is binding to the 1st hyperthread on each core; *not* to both hyperthreads on a single core. :-\

It would still be slightly interesting to see if there's any difference when you run with --bind-to-core instead of paffinity_alone.

On Jan 7, 2011, at 9:56 AM, Gilbert Grosdidier wrote:

> Yes, here it is :
>
> > mpirun -np 8 --mca mpi_paffinity_alone 1 /opt/software/SGI/hwloc/1.1rc6r3028/bin/hwloc-bind --get
> 0x00000001
> 0x00000002
> 0x00000004
> 0x00000008
> 0x00000010
> 0x00000020
> 0x00000040
> 0x00000080
>
> Gilbert.
>
> Le 7 janv. 11 à 15:50, Jeff Squyres a écrit :
>
>> Can you run with np=8?
>>
>> On Jan 7, 2011, at 9:49 AM, Gilbert Grosdidier wrote:
>>
>>> Hi Jeff,
>>>
>>> Thanks for taking care of this.
>>>
>>> Here is what I got on a worker node:
>>>
>>>> mpirun --mca mpi_paffinity_alone 1 /opt/software/SGI/hwloc/1.1rc6r3028/bin/hwloc-bind --get
>>> 0x00000001
>>>
>>> Is this what is expected, please ? Or should I try yet another command ?
>>>
>>> Thanks, Regards, Gilbert.
>>>
>>>
>>>
>>> Le 7 janv. 11 à 15:35, Jeff Squyres a écrit :
>>>
>>>> On Jan 6, 2011, at 11:23 PM, Gilbert Grosdidier wrote:
>>>>
>>>>>> lstopo
>>>>> Machine (35GB)
>>>>> NUMANode L#0 (P#0 18GB) + Socket L#0 + L3 L#0 (8192KB)
>>>>> L2 L#0 (256KB) + L1 L#0 (32KB) + Core L#0
>>>>> PU L#0 (P#0)
>>>>> PU L#1 (P#8)
>>>>> L2 L#1 (256KB) + L1 L#1 (32KB) + Core L#1
>>>>> PU L#2 (P#1)
>>>>> PU L#3 (P#9)
>>>>> L2 L#2 (256KB) + L1 L#2 (32KB) + Core L#2
>>>>> PU L#4 (P#2)
>>>>> PU L#5 (P#10)
>>>>> L2 L#3 (256KB) + L1 L#3 (32KB) + Core L#3
>>>>> PU L#6 (P#3)
>>>>> PU L#7 (P#11)
>>>> [snip]

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/