Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI Blocking Routines and Memory Leaks
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-03-25 14:00:19


Simon Köstlin wrote:

> I'm new to MPI and I've got a question about blocking routines like
> the Send-, Wait-Function and so on. I wrote a parallel program that
> uses the blocking Send and the Nonblocking Isend function. Now my
> question: If I'm sending something with the blocking Send function it
> should block the process until the other process received the message.

No. MPI_Send returns once the message has been taken out of the user's
send buffer. The receiving process might not yet have received the
message. The MPI implementation could have buffered the message
somewhere, and for short messages it probably did. To ensure sync with
receiver, use MPI_Ssend.

> I think that works so far. But I'm also sending a message to the
> process itself and my programm doesn't block.

You're lucky. To send to yourself, it's safest to use nonblocking
operations such as MPI_Irecv/MPI_Send/MPI_Wait (where you wait on the
receive). If you MPI_Send to yourself but there is no receive posted,
an MPI implementation could lock up on you.

> So does MPI not block if I'm sending a message to the same process
> from which I'm sending the message and it is a blocking routine?

That's implementation dependent.

> The same happens if I'm sending with a non-blocking Isend and do a
> request.Wait() on the send request after each send operation. So it
> doesn't block if I'm sending the message to itself. I'm wondering
> about that because the Recv function will occur only after all
> messages have been sent. It's ok that it works, because I need to send
> a message to the process itself for simplicity. I'm only wondering why
> this works.

If the receives are posted only after all the sends, you're relying on
the MPI implementation buffering everything up somewhere. That behavior
is implementation dependent, and typically shorter messages can be
buffered but longer messages will induce sync with the receiver. If you
*really* want to hold off on Recv operations, you might have to buffer
messages yourself with MPI_Bsend. But the generally recommended pattern
is to post MPI_Irecv requests as soon as possible and then post your sends.

> Another question I have is about a memory leak. I got a heavy memory
> leak if I did not a request.Wait() on the send request before the
> Isend function and didn't wait until the last Isend operation
> completed. But all messages were arrived if I do the request.Wait() or
> not. Now I'm doing a request.Wait() before each Isend function and my
> memory isn't increasing much, but still a bit. Do I have to do
> something else on the blocking Send function? And is there a function
> in MPI to clean up its buffers in a running application without using
> the Finalize function.

Doesn't sound right to me, but someone else may understand this better.