Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Interaction of MPI_Send and MPI_Barrier
From: Shaun Jackman (sjackman_at_[hidden])
Date: 2009-07-27 14:17:19


Hi Dick,

Okay, it's good to know that even if using MPI_Barrier in this fashion
did appear to be working, it's not guaranteed to work. Is there an MPI
collective function that has the desired effect? that after all
processes call this function, any previously posted MPI_Send are
guaranteed to have arrived and that MPI_Test will find the message.
Without such a function, it seems it would be impossible to know if
there are any outstanding messages waiting to be received, if it
wasn't known in advance by the receiver to expect the message.
Assuming that each process keeps a counter of the number of messages
that it has sent and the number that it has received, it should be
possible to sum the two counters across all processes and determine if
any are outstanding. It could be accomplished with a single
MPI_Reduce(sent - received).

Cheers,
Shaun

Richard Treumann wrote:
> No - it is not guaranteed. (it is highly probable though)
>
> The return from the MPI_Send only guarantees that the data is safely
> held somewhere other than the send buffer so you are free to modify the
> send buffer. The MPI standard does not say where the data is to be held.
> It only says that once the MPI_Test is successful, the data will have
> been delivered to the receive buffer.
>
> Consider this possible scenario:
>
> MPI_Send is for a small message
> The data is sent toward the destination
> To allow the MPI_Send to complete promptly ,lib MPI makes a temporary
> copy of the message
> The MPI_Send returns once the copy is made
> the message gets lost in the network
> the MPI_Barrier does its communication without packet loss and completes
> the call to MPI_Test returns false
> the send side gets no ack for the lost message and eventually
> retransmits it from the saved temp
> This time it gets through
> A later MPI_Test succeeds
> An ack eventually gets back to the sender and it throws away the temp
> copy of the message it was keeping in case a retransmit was needed
>
> I am not saying any particular MPI library would work this way but it is
> one kind of thing that a libmpi might do to give better performance
> while maintaining the strict rules of MPI semantic. Since the
> MPI_Barrier does not make any guarantee about the destination status of
> sends done before it, this kind of optimization is legitimate.
>
> If you must know that the message is received once the barrier returns,
> you need to MPI_Wait the message before you call barrier.
>
> Dick
>
>
> > Hi,
> >
> > Two processes run the following program:
> >
> > request = MPI_Irecv
> > MPI_Send (to the other process)
> > MPI_Barrier
> > flag = MPI_Test(request)
> >
> > Without the barrier, there's a race and MPI_Test may or may not return
> > true, indicating whether the message has been received. With the
> > barrier, is it guaranteed that the message will have been received,
> > and MPI_Test will return true?
> >
> > Cheers,
> > Shaun