The mailing list snipped off the end of my mail -- here's the rest of
what I said:
The meanings of the 3 phases are explained in this pager: http://
If you use the mpi_leave_pinned parameter and Open MPI is able to
leave your entire buffer registered, it should give the best
performance in OMPI v1.2.4 because it should be able to do a single
RDMA PUT to transfer the majority of the message (after the first
time, of course).
On Oct 12, 2007, at 11:31 AM, Jeff Squyres wrote:
> On Oct 12, 2007, at 8:38 AM, Neeraj Chourasia wrote:
>> Yes, the buffer was being re-used. No we didnt try to benchmark it
>> with netpipe and other stuffs. But the program was pretty simple.
>> Do you think, I need to test it with bigger chunks (>8MB) for
>> We also tried manipulating eager_limit and min_rdma_sze, but no
> It really depends on exactly what your application is doing. If
> you re-use your buffers, then mpi_leave_pinned should help at least
> some. But if the cost of communication is dwarfed by the rest of
> the app, then all this tuning won't make much of a difference.
> Here's a picture of Open MPI's MCA parameters w.r.t. the openib BTL
> and the ob1 PML (we changed some names and ordering between the
> v1.2 series and what will become the v1.3 series -- both are shown
> below) -- hopefully this will help you understand what those MCA
> parameters are for: