thanks for all your replies. 

The master node can receive message ( the same size)  from 50 worker nodes. 
But, it cannot receive message from 51 nodes. It caused "truncate error". 

I used the same buffer to get the message in 50 node case. 

About ""rendezvous" protocol", what is the meaning of "the sender sends a short portion "?

What is the "short portion", is it a small mart of the message of the sender ?
This "rendezvous" protocol" can work automatically in background without programmer
indicates in his program ? 

The "acknowledgement " can be generated by the receiver only when the
corresponding mpi_irecv is posted by the receiver ? 

Any help is appreciated. 


July 10  2010  

Date: Sat, 10 Jul 2010 20:41:26 -0700
From: eugene.loh@oracle.com
To: users@open-mpi.org
Subject: Re: [OMPI users] OpenMPI how large its buffer size ?

I hope I understand the question properly.

The "truncate error" means that the receive buffer provided by the user was too small to receive the designated message.  That's an error in the user code.

You're asking about some buffering sizes within the MPI implementation.  We can talk about that, but it probably first makes sense to clarify what MPI is doing.  If a sender posts a large send and the receiver has not posted a reply, the MPI implementation is not required to move any data.  In particular, most MPI implementations will use a "rendezvous" protocol in which the sender sends a short portion and then waits for an acknowledgement from the receiver that it is ready to receive the message (and knows into which user buffer to place the received data).  This protocol is used so that the MPI implementation does not have to buffer internally arbitrarily large messages.

So, if you post a large send but no receive, the MPI implementation is probably buffering very little data.  The message won't advance until the receive has been posted.  This means that a blocking MPI_Send will wait and a nonblocking MPI_Isend will return without having done much.

Jack Bryan wrote:

thanks for the program from Jody. 

David indicated the question that I want to ask. 

But, Jody's approach is ok when the MPI built-in buffer size is large enough to hold the 
message such as 100kB in the buffer. 

In asynchronous communication, when the sender posts a mpi_isend, the message is put in 
a buffer provided by the MPI. 

At this point, the receiver may still not post its corresponding mpi_irecv. So, the buffer size is 
important here. 

Without knowing the buffer size, I may get " truncate error " on Open MPI. 

How to know the size of the buffer automatically created by Open MPI in the background ?

Any help is appreciated. 


July 10 2010

From: solarbikedz@gmail.com
Date: Sat, 10 Jul 2010 16:46:12 -0700
To: users@open-mpi.org
Subject: Re: [OMPI users] OpenMPI how large its buffer size ?

I believe his question is regarding when under non-blocking send/recv, how does MPI know how much memory to allocate to receive the message, since the size is determined AFTER the irecv is posted.  So if the send post isend, but the receiver hasn't post irecv, what would the MPI do with the message.

I believe MPI would automatically create a buffer in the background to store the message.

On Sat, Jul 10, 2010 at 1:55 PM, jody <jody.xha@gmail.com> wrote:
Perhaps i misunderstand your question...
Generally, it is the user's job to provide the buffers both to send and receive.
If you call MPI_Recv, you must pass a buffer that is large enough to
hold the data sent by the
corresponding MPI_Send. I.e., if you know your sender will send
messages of 100kB,
then you must provide a buffer of size 100kB to the receiver.
If the message size is unknown at compile time, you may have to send
two messages:
first an integer which tells the receiver how large a buffer it has to
allocate, and then
the actual message (which then nicely fits into the freshly allocated buffer)

#include <stdio.h>
#include <stdlib.h>

#include <time.h>

#include "mpi.h"

#define SENDER     1
#define RECEIVER   0
#define TAG_LEN   77
#define TAG_DATA  78
#define MAX_MESSAGE 16

int main(int argc, char *argv[]) {

   int num_procs;
   int rank;
   int *send_buf;
   int *recv_buf;
   int send_message_size;
   int recv_message_size;
   MPI_Status st;
   int i;

   /* initialize random numbers */
   MPI_Init(&argc, &argv);
   MPI_Comm_size(MPI_COMM_WORLD, &num_procs);
   MPI_Comm_rank(MPI_COMM_WORLD, &rank);

   if (rank == RECEIVER) {
       /* the receiver */
       /* wait for message length */
       MPI_Recv(&recv_message_size, 1, MPI_INT, SENDER, TAG_LEN,
       /* create a buffer of the required size */
       recv_buf = (int*) malloc(recv_message_size*sizeof(int));
       /* get data */
       MPI_Recv(recv_buf, recv_message_size, MPI_INT, SENDER,

       printf("Receiver got %d integers:", recv_message_size);
       for (i = 0; i < recv_message_size; i++) {
           printf(" %d", recv_buf[i]);

       /* clean up */

   } else if (rank == SENDER) {
       /* the sender */
       /* random message size */
       send_message_size = (int)((1.0*MAX_MESSAGE*rand())/(1.0*RAND_MAX));
       /* create a buffer of the required size */
       send_buf = (int*) malloc(send_message_size*sizeof(int));
       /* create random message */
       for (i = 0; i < send_message_size; i++) {
           send_buf[i] = rand();

       printf("Sender has %d integers:", send_message_size);
       for (i = 0; i < send_message_size; i++) {
           printf(" %d", send_buf[i]);

       /* send message size to receiver */
       MPI_Send(&send_message_size,  1, MPI_INT, RECEIVER, TAG_LEN,
       /* now send messagge */
       MPI_Send(send_buf, send_message_size, MPI_INT, RECEIVER,

       /* clean up */



I hope this helps

On Sat, Jul 10, 2010 at 7:12 AM, Jack Bryan <dtustudy68@hotmail.com> wrote:
> Dear All:
> How to find the buffer size of OpenMPI ?
> I need to transfer large data between nodes on a cluster with OpenMPI 1.3.4.
> Many nodes need to send data to the same node .
> Workers use mpi_isend, the receiver node use  mpi_irecv.
> because they are non-blocking, the messages are stored in buffers of
> senders.
> And then, the receiver collect messages from its buffer.
> If the receiver's buffer is too small, there will be truncate error.
> Any help is appreciated.

Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox. Learn more.