Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI_Allreduce hangs
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-06-27 14:30:11


On Jun 27, 2012, at 2:25 PM, Martin Siegert wrote:

>> http://www.open-mpi.org/~jsquyres/unofficial/openmpi-1.6.1ticket3131r26612M.tar.bz2
>
> Thanks! I tried this and, indeed, the program (I tested quantum espresso,
> pw.x, so far) no longer hangs.

Good! We're doing a bit more definitive testing here (took a little while to figure out how to do that, but we're in process of doing that now...) before we let this go out into the wild.

> Then I went one step further and benchmarked the following three cases:
>
> 1) pw.x compiled with openmpi-1.3.3
> 2) pw.x compiled with openmpi-1.4.3 and
> btl_openib_flags = 305
> btl_openib_eager_limit = 65536
> in etc/openmpi-mca-params.conf
> 3) pw.x compiled with openmpi-1.6.1ticket3131r26612M
>
> These are the results time (in seconds) per iteration - smaller is better:
> 1) 33.11
> 2) 28.23
> 3) 34.81
>
> That's rather disappointing, isn't it?

Yes, it is. But #2 is not really comparable with #1 and #3. It's quite possible that with newer IB hardware, the eager limit should be bumped up by default.

I leave this to Mellanox to figure out...

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/