Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] [EXTERNAL] Re: Configuration for rendezvous and eager protocols: two-sided comm
From: Shamis, Pavel (shamisp_at_[hidden])
Date: 2013-12-26 11:19:59


Hey Sid ;)
Please see inline.

1.1. What is the meaning of "phase 1" fragment?
[Shamis, Pavel] Phase one of the rendezvous (RNDV) protocol. Essentially, the first request of the RNDV we send as an eager message of size N,
Receiver unpacks the messages and switches RNDV get or put semantics.

1.2. Is my understanding correct that the btl_*_eager_limit is applicable only in case of one-sided communication?

[Shamis, Pavel] It is actually used for small messages in the send-receive semantics. The idea is that the cost of memory registration is relatively high,

To avoid it, we memcopy the data to a pre-registred /allocated buffer.

Best,

P.

On 16 December 2013 14:36, Jeff Squyres (jsquyres) <jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>> wrote:
Everything that Brian said, plus: note that the MCA param that Christoph mentioned is specifically for the "sm" (shared memory) transport. Each transport has their own set of MCA params (e.g., mca_btl_tcp_eager_limit, and friends).

On Dec 16, 2013, at 3:19 PM, "Barrett, Brian W" <bwbarre_at_[hidden]<mailto:bwbarre_at_[hidden]>> wrote:

> Siddhartha -
>
> Christoph mentioned how to change the cross-over for shared memory, but it's really per-transport (so you'd have to change it for your off-node transport as well). That's all in the FAQ you mentioned, so hopefully you can take it from there. Note that, in general, moving the eager limits has some unintended side effects. For example, it can cause more / less copies. It can also greatly increase memory usage.
>
> Good luck,
>
> Brian
>
> On 12/16/13 1:49 AM, "Siddhartha Jana" <siddharthajana24_at_[hidden]<mailto:siddharthajana24_at_[hidden]>> wrote:
>
>> Thanks Christoph.
>> I should have looked into the FAQ section on MCA params setting @ :
>> http://www.open-mpi.org/faq/?category=tuning#available-mca-params
>>
>> Thanks again,
>> -- Siddhartha
>>
>>
>> On 16 December 2013 02:41, Christoph Niethammer <niethammer_at_[hidden]<mailto:niethammer_at_[hidden]>> wrote:
>>> Hi Siddhartha,
>>>
>>> MPI_Send/Recv in Open MPI implements both protocols and chooses based on the message size which one to use.
>>> You can use the mca parameter "btl_sm_eager_limit" to modify the behaviour.
>>>
>>> Here the corresponding info obtained from the ompi_info tool:
>>>
>>> "btl_sm_eager_limit" (current value: <4096>, data source: default value)
>>> Maximum size (in bytes) of "short" messages (must be >= 1)
>>>
>>> Regards
>>> Christoph Niethammer
>>>
>>> --
>>>
>>> Christoph Niethammer
>>> High Performance Computing Center Stuttgart (HLRS)
>>> Nobelstrasse 19
>>> 70569 Stuttgart
>>>
>>> Tel: ++49(0)711-685-87203<tel:%2B%2B49%280%29711-685-87203>
>>> email: niethammer_at_[hidden]<mailto:niethammer_at_[hidden]>
>>> http://www.hlrs.de/people/niethammer
>>>
>>>
>>>
>>> ----- Ursprüngliche Mail -----
>>> Von: "Siddhartha Jana" <siddharthajana24_at_[hidden]<mailto:siddharthajana24_at_[hidden]>>
>>> An: "OpenMPI users mailing list" <users_at_[hidden]<mailto:users_at_[hidden]>>
>>> Gesendet: Samstag, 14. Dezember 2013 13:44:12
>>> Betreff: [OMPI users] Configuration for rendezvous and eager protocols: two-sided comm
>>>
>>>
>>>
>>> Hi
>>>
>>>
>>> In OpenMPI, are MPI_Send, MPI_Recv (and friends) implemented using rendezvous protocol or eager protocol?
>>>
>>>
>>> If both, is there a way to choose one or the other during runtime or while building the library?
>>>
>>>
>>> If there is a threshold of the message size that dictates the protocol to be used, is there a way I can alter that threshold value?
>>>
>>>
>>> If different protocols were used for different versions of the library in the past, could someone please direct me to the exact version numbers of the implementations that used one or the other protocol?
>>>
>>>
>>> Thanks a lot,
>>> Siddhartha
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]<mailto:users_at_[hidden]>
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]<mailto:users_at_[hidden]>
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>
>
> --
> Brian W. Barrett
> Scalable System Software Group
> Sandia National Laboratories
> _______________________________________________
> users mailing list
> users_at_[hidden]<mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users

--
Jeff Squyres
jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
_______________________________________________
users mailing list
users_at_[hidden]<mailto:users_at_[hidden]>
http://www.open-mpi.org/mailman/listinfo.cgi/users