Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] A question on MPI_Probe
From: Iliev, Hristo (iliev_at_[hidden])
Date: 2012-10-01 04:12:04


I believe Jeff Squyres has already answered your question on the Open MPI
mailing list, or at least hinted on the possible problem. MPI messages are
received in the order they were sent but only within the specific (tag,
communicator) tuple. This basically means:

- within the same communicator you can receive messages out of order if they
carry different tags;

- messages that carry the same tag can be received out of order if they were
communicated in different contexts (communicators).

But here is the catch: you cannot receive a message that has not been sent
yet. If you have two consecutive send operations, you must make sure that the
first one would not block forever. The standard MPI send operation MPI_Send
could be implemented in various ways (the standard doesn't say exactly how),
but in most MPI implementations it behaves like buffered send for very small
messages and like synchronous send for larger messages. If you have the
following two calls in your sender process:

MPI_Send(largedata, largecount, MPI_INT, dest, tag1, MPI_COMM_WORLD);

MPI_Send(smalldata, smallcount, MPI_INT, dest, tag2, MPI_COMM_WORLD);

it could happen that the first MPI_Send would actually behave as a synchronous
one, i.e. it would not return unless the matching receive operation was posted
on the receiver's side. Suppose that your receiver code is:

MPI_Probe(src, tag2, MPI_COMM_WORLD, &status);

MPI_Recv(largedata, largecount, MPI_INT, src, tag1, MPI_COMM_WORLD,

This would most likely deadlock because MPI_Probe is a blocking call, i.e. it
would not return until a matching send was posted, i.e. the second MPI_Send
would have to execute, which would only happen after the first send has
returned, but it would not happen unless the MPI_Recv in the receiver is
executed... I guess you get the idea.

To prevent the deadlock you could modify the sender's code to use a
non-blocking send:

MPI_Request req;

MPI_Isend(largedata, largecount, MPI_INT, dest, tag1, MPI_COMM_WORLD, &req);

MPI_Send(smalldata, smallcount, MPI_INT, dest, tag2, MPI_COMM_WORLD);


Using non-blocking operation the send call returns immediately and the
operation continues in the background, so the second send would get executed
immediately after that. Now there would be two pending messages and they can
be received in any order since they carry different tags.

Hope that helps!

Best regards,


Hristo Iliev, Ph.D. -- High Performance Computing
RWTH Aachen University, Center for Computing and Communication
Rechen- und Kommunikationszentrum der RWTH Aachen
Seffenter Weg 23,  D 52074  Aachen (Germany)
Tel: +49 241 80 24367 -- Fax/UMS: +49 241 80 624367
From: users-bounces_at_[hidden] [mailto:users-bounces_at_[hidden]] On Behalf 
Of devendra rai
Sent: Saturday, September 22, 2012 12:33 AM
To: Open Users
Subject: [OMPI users] A question on MPI_Probe
I believe my understanding of MPI_Probe is not correct. Here's what I have as 
a setup:
Two MPI processes, A and B. Process A sends a large message, M1 using tag T1, 
and a small message, M2 using tag T2. The recepient of both messages is B.
The order of sending the messages is M1 and then M2.
The process B however executes MPI_Probe to test if the message with tag T2 is 
available. B will accept M1 after it has received M2 first.
I am under the impression that using information gained from MPI_Probe, I can 
choose not to receive message M1, and instead look for M2.
However, I see that M2 is never received by B (although A confirms that both 
M1 and M2 have been sent).
I am little confused. Can someone explain why B cannot receive M2? At least, 
does MPI allow receiving messages in the order that I have just described?
Thanks a lot.
Devendra Rai.

  • application/pkcs7-signature attachment: smime.p7s