Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] max. message size
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2013-07-18 07:34:26


I'm assuming that you are referring to the fact that MPI uses an "int" to specify the "length" of a message (i.e., the number of items to be sent and received), like this:

  MPI_Send(buffer, count, datatype, ...)

"count" is an int, which means that it has a max value of INT_MAX, which, on today's 64 bit machines, is about 2 billion.

Note, however, that the message that is sent is really "count times length of datatype", and datatype can represent something more than 1 byte. For example, if you use (count=2B, datatype=MPI_INT), that represents ~2 billion MPI_INTs, and an "int" is typically 4 bytes, so you're sending an ~8 billion byte message.

And you can use the MPI datatype constructors to create a datatype that is longer than that (google around; you'll find some good MPI datatype constructor tutorials). E.g., you can send (count=2B, datatype=my_datatype), where my_datatype represents, say, 128 bytes. Then you'll be sending a message that is (2 billion * 128) bytes.

And so on.

So there's two typical ways to send "extra large" messages in MPI:

1. Use a (count, datatype) tuple that will represent more than 2 billion bytes.

2. Break your message up into multiple parts. E.g., if you have 16GB message to send, send it in 16 1GB chunks (which I assume is what Changsheng Jiang's code snipit was doing).

On Jul 18, 2013, at 4:54 AM, mohammad assadsolimani <m.assadsolimani_at_[hidden]> wrote:

> Dear Mike,
>
> Thank you for your response. I use this program
> http://reduze.hepforge.org/
>
> I did not rite its code myself.Did I understand you right?
> You mean I have to use the new version of OMPI namely:
>
> OMPI 1.7
>
> because the version which I use is: mpirun (Open MPI) 1.5.5
> What is IB ?
>
> thank you in advanced
>
> Cheers,
> Mohammad
>
>
>>
>> Message: 5
>> Date: Wed, 17 Jul 2013 16:15:41 +0300
>> From: Mike Dubman <miked_at_[hidden]>
>> Subject: Re: [OMPI users] max. message size
>> To: Open MPI Users <users_at_[hidden]>
>> Message-ID:
>> <CAAO1KyakymR7wqOMwW31ntZrbYShMzkhh9Wvm7yy1Swofmh=1w_at_[hidden]>
>> Content-Type: text/plain; charset="iso-8859-1"
>>
>> do you use IB as a transport? max message size in IB/RDMA is limited to
>> 2G, but OMPI 1.7 splits large buffers during RDMA into 2G chunks.
>>
>>
>>
>> On Wed, Jul 17, 2013 at 11:51 AM, mohammad assadsolimani <
>> m.assadsolimani_at_[hidden]> wrote:
>>
>>>

Post removed!








>>> ______________________________**_________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/**mailman/listinfo.cgi/users>
>>>
>> -------------- next part --------------
>> HTML attachment scrubbed and removed
>>
>> ------------------------------
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>>
http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> End of users Digest, Vol 2631, Issue 1
>> **************************************
>>
>
>
>
> --------------------------------------------------------------
> Webmail: http://mail.livenet.ch
> Glauben entdecken: http://www.jesus.ch
> Christliches Webportal: http://www.livenet.ch
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/