Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] UDP like messaging with MPI
From: Lukas Razik (linux_at_[hidden])
Date: 2011-11-19 12:42:43


>I know about tnıs functıons, they special requirements  like the mpi_irecv call should be made in every process. My processes should not look for messages or implicitly receive them.

I understand. But then I think your UDP comparison is wrong - whatever... :)

> But messages shuddering go into their msg queues and retrieved when needed.

I don't know if MPI has something like message queues. If not then I would use "MPI RMA" (i.e. MPI Remote Memory Access from MPI-2) and implement a message queue like this:

1. On the node which shall get the messages (I call it G) I would create a buffer with a size which is as big as needed to store enough messages from the processes which shall write into the message queue.
2. With the collective operation MPI_Win_create() I would make the buffer of G available for all participating nodes.
3. Now the sender nodes can put data into the window buffer with MPI_Put():
Here the problem is that they must know which part of the buffer is already overwritten with messages of other senders.
Therefore I would use a integer value at the beginning of the window buffer. This value shows the next free position in the window buffer.
In other words each sender who wants to save a message in the window buffer
 a) reads the integer value from the buffer

 b) saves its message at the free place in the buffer
 c) increments the integer value by the size of the written message
4. To avoid race conditions you must lock the accesses to the window by MPI_Win_lockI() and MPI_Win_unlock().
BTW: There are three type of synchronization calls:
 a) MPI_Win_fence()
 b) MPI_Win_start(), MPI_Win_complete, MPI_Win_post(), MPI_Win_wait()
 c) MPI_Win_lock(), MPI_Win_unlock()But I think the right one here is c) because then the target process of your MPI_put() (i.e. G) doesn't need to be involved in the communication. Therefore a communication which is synchronized by method c) is called a "passive target communication". Have a look into the MPI-2 standard if you don't know what that means.

5. When G wants to "receive" from the windows buffer it must also call the MPI_Win_lock() & MPI_Win_unlock() operations. Then it reads one or more messages from the buffer window. After that it must decrement the integer value by the size of the read message(s).

This would be a LIFO message queue. If you want a FIFO queue then implement a "ring buffer". Therefore you could use two integer values at the beginning of the buffer which show the head and the tail of the queue. Maybe there's also an more efficient way but that's an idea I have.

> Just like udp communication.

In the best of my knowledge "normal" UDP sockets have no receive queues . So if there's no receiver which waits for an incoming UDP datagram then it's discarded, isn't it? (Maybe asynchronous UDP sockets have queues...)