Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] btl_openib_cpc_include rdmacm questions
From: Ralph Castain (rhc_at_[hidden])
Date: 2011-04-21 18:49:26


On Apr 21, 2011, at 4:41 PM, Brock Palen wrote:

> Given that part of our cluster is TCP only, openib wouldn't even startup on those hosts

That is correct - it would have no impact on those hosts

> and this would be ignored on hosts with IB adaptors?

Ummm...not sure I understand this one. The param -will- be used on hosts with IB adaptors because that is what it is controlling.

However, it -won't- have any impact on hosts without IB adaptors, which is what I suspect you meant to ask?

>
> Just checking thanks!
>
> Brock Palen
> www.umich.edu/~brockp
> Center for Advanced Computing
> brockp_at_[hidden]
> (734)936-1985
>
>
>
> On Apr 21, 2011, at 6:21 PM, Jeff Squyres wrote:
>
>> Over IB, I'm not sure there is much of a drawback. It might be slightly slower to establish QP's, but I don't think that matters much.
>>
>> Over iWARP, rdmacm can cause connection storms as you scale to thousands of MPI processes.
>>
>>
>> On Apr 20, 2011, at 5:03 PM, Brock Palen wrote:
>>
>>> We managed to have another user hit the bug that causes collectives (this time MPI_Bcast() ) to hang on IB that was fixed by setting:
>>>
>>> btl_openib_cpc_include rdmacm
>>>
>>> My question is if we set this to the default on our system with an environment variable does it introduce any performance or other issues we should be aware of?
>>>
>>> Is there a reason we should not use rdmacm?
>>>
>>> Thanks!
>>>
>>> Brock Palen
>>> www.umich.edu/~brockp
>>> Center for Advanced Computing
>>> brockp_at_[hidden]
>>> (734)936-1985
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> --
>> Jeff Squyres
>> jsquyres_at_[hidden]
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users