Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Anyscientific application heavily using MPI_Barrier?
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2009-08-24 15:40:15

On Aug 24, 2009, at 1:03 PM, Eugene Loh wrote:

> E.g., let's say P0 and P1 each send a message to P2, both using the
> same tag and communicator. Let's say P2 does two receives on that
> communicator and tag, using a wildcard source. So, the messages
> could be received in either order. One could introduce barriers to
> order the messages. E.g.,
> P0:
> Send
> Barrier
> P1:
> Barrier
> Send
> P2:
> Recv
> Barrier
> Recv

Is this behavior *guaranteed* by MPI? I'm not actually sure that it
is; barrier does not provide any guarantees about point-to-point
message passing progress.

For example, how about a machine with these assumptions:

- P0 is "far away" from P2 on the point-to-point network
- P1 is "close by" to P2 on the point-to-point network
- Barriers go across a separate/fast network (think: bluegene)
- P0's send message is short/eager

In this case, the Send from P0 complete "immediately" and enter the
barrier before it is delivered to P2. The P0 send could then take a
"long time" to get to P2 -- potentially long enough for the barrier to
overtake it and for the Send from P1 to be delivered to P2 before the
Send from P0 arrives at P2.

Couldn't that happen?

Granted, I would expect that your example would perform in most real-
world situations as you describe (P0 is delivered to P2, then P1 is
delivered to P2). But I don't think the standard guarantees it.

Jeff Squyres