Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-10-12 11:38:13

The mailing list snipped off the end of my mail -- here's the rest of
what I said:

The meanings of the 3 phases are explained in this pager: http://

If you use the mpi_leave_pinned parameter and Open MPI is able to
leave your entire buffer registered, it should give the best
performance in OMPI v1.2.4 because it should be able to do a single
RDMA PUT to transfer the majority of the message (after the first
time, of course).

On Oct 12, 2007, at 11:31 AM, Jeff Squyres wrote:

> On Oct 12, 2007, at 8:38 AM, Neeraj Chourasia wrote:
>> Yes, the buffer was being re-used. No we didnt try to benchmark it
>> with netpipe and other stuffs. But the program was pretty simple.
>> Do you think, I need to test it with bigger chunks (>8MB) for
>> communication.?
>> We also tried manipulating eager_limit and min_rdma_sze, but no
>> success.
> It really depends on exactly what your application is doing. If
> you re-use your buffers, then mpi_leave_pinned should help at least
> some. But if the cost of communication is dwarfed by the rest of
> the app, then all this tuning won't make much of a difference.
> Here's a picture of Open MPI's MCA parameters w.r.t. the openib BTL
> and the ob1 PML (we changed some names and ordering between the
> v1.2 series and what will become the v1.3 series -- both are shown
> below) -- hopefully this will help you understand what those MCA
> parameters are for:
> <ob1-params.pdf>
> <mime-attachment.txt>
> <mime-attachment.txt>

Jeff Squyres
Cisco Systems