Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Mixed Mellanox and Qlogic problems
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2011-07-15 12:03:29


I don't think too many people have done combined QLogic + Mellanox runs, so this probably isn't a well-explored space.

Can you run some microbenchmarks to see what kind of latency / bandwidth you're getting between nodes of the same type and nodes of different types?

On Jul 14, 2011, at 8:21 PM, David Warren wrote:

> On my test runs (wrf run just long enough to go beyond the spinup influence)
> On just 6 of the the old mlx4 machines I get about 00:05:30 runtime
> On 3 mlx4 and 3 qib nodes I get avg of 00:06:20
> So the slow down is about 11+%
> When this is a full run 11% becomes a evry long time. This has held for some longer tests as well before I went to ofed 1.6.
>
> On 07/14/11 05:55, Jeff Squyres wrote:
>> On Jul 13, 2011, at 7:46 PM, David Warren wrote:
>>
>>
>>> I finally got access to the systems again (the original ones are part of our real time system). I thought I would try one other test I had set up first. I went to OFED 1.6 and it started running with no errors. It must have been an OFED bug. Now I just have the speed problem. Anyone have a way to make the mixture of mlx4 and qlogic work together without slowing down?
>>>
>> What do you mean by "slowing down"?
>>
>>
> <warren.vcf>

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/