Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] max. message size
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2013-07-18 07:34:26


I'm assuming that you are referring to the fact that MPI uses an "int" to specify the "length" of a message (i.e., the number of items to be sent and received), like this:

  MPI_Send(buffer, count, datatype, ...)

"count" is an int, which means that it has a max value of INT_MAX, which, on today's 64 bit machines, is about 2 billion.

Note, however, that the message that is sent is really "count times length of datatype", and datatype can represent something more than 1 byte. For example, if you use (count=2B, datatype=MPI_INT), that represents ~2 billion MPI_INTs, and an "int" is typically 4 bytes, so you're sending an ~8 billion byte message.

And you can use the MPI datatype constructors to create a datatype that is longer than that (google around; you'll find some good MPI datatype constructor tutorials). E.g., you can send (count=2B, datatype=my_datatype), where my_datatype represents, say, 128 bytes. Then you'll be sending a message that is (2 billion * 128) bytes.

And so on.

So there's two typical ways to send "extra large" messages in MPI:

1. Use a (count, datatype) tuple that will represent more than 2 billion bytes.

2. Break your message up into multiple parts. E.g., if you have 16GB message to send, send it in 16 1GB chunks (which I assume is what Changsheng Jiang's code snipit was doing).

On Jul 18, 2013, at 4:54 AM, mohammad assadsolimani <m.assadsolimani_at_[hidden]> wrote:

> Dear Mike,
>
> Thank you for your response. I use this program
> http://reduze.hepforge.org/
>
> I did not rite its code myself.Did I understand you right?
> You mean I have to use the new version of OMPI namely:
>
> OMPI 1.7
>
> because the version which I use is: mpirun (Open MPI) 1.5.5
> What is IB ?
>
> thank you in advanced
>
> Cheers,
> Mohammad
>
>
>>
>> Message: 5
>> Date: Wed, 17 Jul 2013 16:15:41 +0300
>> From: Mike Dubman <miked_at_[hidden]>
>> Subject: Re: [OMPI users] max. message size
>> To: Open MPI Users <users_at_[hidden]>
>> Message-ID:
>> <CAAO1KyakymR7wqOMwW31ntZrbYShMzkhh9Wvm7yy1Swofmh=1w_at_[hidden]>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> do you use IB as a transport? max message size in IB/RDMA is limited to
>> 2G, but OMPI 1.7 splits large buffers during RDMA into 2G chunks.
>>
>>
>>
>> On Wed, Jul 17, 2013 at 11:51 AM, mohammad assadsolimani <
>> m.assadsolimani_at_[hidden]> wrote:
>>
>>>

Post removed!








>>> ______________________________**_________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/**mailman/listinfo.cgi/users>
>>>
>> -------------- next part --------------
>> HTML attachment scrubbed and removed
>>
>> ------------------------------
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>>
http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> End of users Digest, Vol 2631, Issue 1
>> **************************************
>>
>
>
>
> --------------------------------------------------------------
> Webmail: http://mail.livenet.ch
> Glauben entdecken: http://www.jesus.ch
> Christliches Webportal: http://www.livenet.ch
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/