On Aug 27, 2008, at 10:22 AM, Richard Treumann wrote:

Robert -

A return from a blocking send means the application send buffer is available for reuse. If it is a BSEND, the application buffer could be available because the message data has been copied to the attached buffer or because the data has been delivered to the destination. The application on the side that used BSEND only cares that the application send buffer is available and a receive on the destination side can get the data from somewhere.

A return from a blocking receive means the data is available in the application receive buffer. If there were a BRECV in the standard, what would it mean? It cannot return until the data is in the application receive buffer


I see, not it's clear.

However from an application point of view I see an odd result here. On the sender side I can use a buffered send to "queue" messages for delivery and decide how many messages my buffer should contain before the MPI_Bsend blocks if it's running out of space.

On the receiving side I have no control over the number of messages that MPI can buffer. This is basically left to the implementation details as you very well described. Shouldn't the user be allowed to specify a memory space to buffer messages on the receiving side, just like on the send?

so at the same time the BRECV call is able to tell libmpi to "use a scratch buffer", libmpi is also being told the address of the application buffer and would have no use for a scratch buffer. Before the BRECV call, libmpi could not know if the message will be matched by BRECV or some other kind of receive.

There is a hidden optimization in most MPI implementations that will allow small sends to ship their data to the destination eagerly. When the data shows up at the destination and there is no matching receive, the data is buffered in space hidden within the MPI implementation. When a receive call is made, libmpi will first check to see if the data is already available in this hidden scratch buffer. In this optimization, when there is a matching receive waiting for the data at the moment it arrives, the hidden buffering is bypassed.

There is nothing in the MPI API to let an application say a receive or send should use this hidden buffering. That is up to libmpi.

There is a ready send defined by the MPI standard which allows the send side to dictate that the message go immediately and libmpi need not attempt to buffer it. If the application can be certain the matching receive is posted before the RSEND call is made, there is a possibility of some performance gains but pre-posting receives with absolute reliability is not as easy as you might expect.

The danger with RSEND is that if the receive is not available the data must be discarded. Since the MPI standard does not allow data to be lost, the next thing the MPI library will do after discarding the data is abort the job.


Dick Treumann - MPI Team
IBM Systems & Technology Group
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363


users-bounces@open-mpi.org wrote on 08/27/2008 08:41:59 AM:

> [image removed]

>
> Re: [OMPI users] MPI_Brecv vs multiple MPI_Irecv

>
> Robert Kubrick

>
> to:

>
> Open MPI Users

>
> 08/27/2008 08:51 AM

>
> Sent by:

>
> users-bounces@open-mpi.org

>
> Cc:

>
> mpich-discuss

>
> Please respond to Open MPI Users

>
> Interesting, MPI_Brecv is not in the standard (MPI_Bsend is). I don't  
> understand why a recv could not implemented with an external buffer.
> Anyway, my bad, I was assuming MPI_Brecv was on. Thanks George for  
> pointing this out.
>
> On Aug 27, 2008, at 7:14 AM, George Bosilca wrote:
>
> > What is the definition of a buffered receive ?
> >
> >   george.
> >
> > On Aug 26, 2008, at 10:17 PM, Robert Kubrick wrote:
> >
> >> From a performance point of view, which one is better:
> >>
> >> MPI_Battach(10*sizeof(MSG))
> >> MPI_Brecv()
> >>
> >> or
> >>
> >> MPI_recv_init()
> >> MPI_recv_init()
> >> MPI_recv_init()
> >> ... /* 10 recv handlers */
> >> MPI_Start(all recv)
> >> MPI_Waitany()
> >>
> >>
> >> I understand MPI_Brecv will require an extra message copy, from  
> >> the attached buffer to the MPI_Brecv() buffer. I'd like to know if  
> >> there other differences between the two methods.
> >>
> >> Thanks,
> >> Rob
> >> _______________________________________________
> >> users mailing list
> >> users@open-mpi.org
> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> > _______________________________________________
> > users mailing list
> > users@open-mpi.org
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users

_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users