Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI_Brecv vs multiple MPI_Irecv
From: Robert Kubrick (robertkubrick_at_[hidden])
Date: 2008-08-27 11:55:58


On Aug 27, 2008, at 10:22 AM, Richard Treumann wrote:

> Robert -
>
> A return from a blocking send means the application send buffer is
> available for reuse. If it is a BSEND, the application buffer could
> be available because the message data has been copied to the
> attached buffer or because the data has been delivered to the
> destination. The application on the side that used BSEND only cares
> that the application send buffer is available and a receive on the
> destination side can get the data from somewhere.
>
> A return from a blocking receive means the data is available in the
> application receive buffer. If there were a BRECV in the standard,
> what would it mean? It cannot return until the data is in the
> application receive buffer
>

I see, not it's clear.

However from an application point of view I see an odd result here.
On the sender side I can use a buffered send to "queue" messages for
delivery and decide how many messages my buffer should contain before
the MPI_Bsend blocks if it's running out of space.

On the receiving side I have no control over the number of messages
that MPI can buffer. This is basically left to the implementation
details as you very well described. Shouldn't the user be allowed to
specify a memory space to buffer messages on the receiving side, just
like on the send?

> so at the same time the BRECV call is able to tell libmpi to "use a
> scratch buffer", libmpi is also being told the address of the
> application buffer and would have no use for a scratch buffer.
> Before the BRECV call, libmpi could not know if the message will be
> matched by BRECV or some other kind of receive.
>
> There is a hidden optimization in most MPI implementations that
> will allow small sends to ship their data to the destination
> eagerly. When the data shows up at the destination and there is no
> matching receive, the data is buffered in space hidden within the
> MPI implementation. When a receive call is made, libmpi will first
> check to see if the data is already available in this hidden
> scratch buffer. In this optimization, when there is a matching
> receive waiting for the data at the moment it arrives, the hidden
> buffering is bypassed.
>
> There is nothing in the MPI API to let an application say a receive
> or send should use this hidden buffering. That is up to libmpi.
>
> There is a ready send defined by the MPI standard which allows the
> send side to dictate that the message go immediately and libmpi
> need not attempt to buffer it. If the application can be certain
> the matching receive is posted before the RSEND call is made, there
> is a possibility of some performance gains but pre-posting receives
> with absolute reliability is not as easy as you might expect.
>
> The danger with RSEND is that if the receive is not available the
> data must be discarded. Since the MPI standard does not allow data
> to be lost, the next thing the MPI library will do after discarding
> the data is abort the job.
>
>
> Dick Treumann - MPI Team
> IBM Systems & Technology Group
> Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
> Tele (845) 433-7846 Fax (845) 433-8363
>
>
> users-bounces_at_[hidden] wrote on 08/27/2008 08:41:59 AM:
>
> > [image removed]
> >
> > Re: [OMPI users] MPI_Brecv vs multiple MPI_Irecv
> >
> > Robert Kubrick
> >
> > to:
> >
> > Open MPI Users
> >
> > 08/27/2008 08:51 AM
> >
> > Sent by:
> >
> > users-bounces_at_[hidden]
> >
> > Cc:
> >
> > mpich-discuss
> >
> > Please respond to Open MPI Users
> >
> > Interesting, MPI_Brecv is not in the standard (MPI_Bsend is). I
> don't
> > understand why a recv could not implemented with an external buffer.
> > Anyway, my bad, I was assuming MPI_Brecv was on. Thanks George for
> > pointing this out.
> >
> > On Aug 27, 2008, at 7:14 AM, George Bosilca wrote:
> >
> > > What is the definition of a buffered receive ?
> > >
> > > george.
> > >
> > > On Aug 26, 2008, at 10:17 PM, Robert Kubrick wrote:
> > >
> > >> From a performance point of view, which one is better:
> > >>
> > >> MPI_Battach(10*sizeof(MSG))
> > >> MPI_Brecv()
> > >>
> > >> or
> > >>
> > >> MPI_recv_init()
> > >> MPI_recv_init()
> > >> MPI_recv_init()
> > >> ... /* 10 recv handlers */
> > >> MPI_Start(all recv)
> > >> MPI_Waitany()
> > >>
> > >>
> > >> I understand MPI_Brecv will require an extra message copy, from
> > >> the attached buffer to the MPI_Brecv() buffer. I'd like to
> know if
> > >> there other differences between the two methods.
> > >>
> > >> Thanks,
> > >> Rob
> > >> _______________________________________________
> > >> users mailing list
> > >> users_at_[hidden]
> > >> http://www.open-mpi.org/mailman/listinfo.cgi/users
> > >
> > > _______________________________________________
> > > users mailing list
> > > users_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users