Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Brian Barrett (brbarret_at_[hidden])
Date: 2006-03-10 08:17:40


On Mar 10, 2006, at 2:24 AM, Troy Telford wrote:

> On Mar 9, 2006, at 9:18 PM, Brian Barrett wrote:
>
>> On Mar 9, 2006, at 6:41 PM, Troy Telford wrote:
>>
>>> I've got a machine that has the following config:
>>>
>>> Each node has two InfiniBand ports:
>>> * The first port is on fabric 'a' with switches for 'a'
>>> * The second port is on fabric 'b' with separate switches for 'b'
>>> * The two fabrics are not shared ('a' and 'b' can't communicate
>>> with one
>>> another)
>>>
>>> I believe that Open MPI is perfectly capable of stripeing over both
>>> fabric
>>> 'a' and 'b', and IIRC, this is the default behavior.
>>>
>>> Does Open MPI handle the case where Open MPI puts all of its
>>> traffic on
>>> the first IB port (ie. fabric 'a'), and leaves the second IB port
>>> (ie.
>>> fabric 'b') free for other uses (I'll use NFS as a humorous
>>> example).
>>>
>>> If so, is there any magic required to configure it thusly?
>>
>> With mvapi, we don't have the functionality in place for the user to
>> specify which HCA port is used. The user can say that at most N HCA
>> ports should be used through the btl_mvapi_max_btls MCA parameter.
>> So in your case, if you ran Open MPI with:
>>
>> mpirun -mca btl_mvapi_max_btls 1 -np X ./foobar
>>
>> Only the first active port would be used for mvapi communication.
>> I'm not sure if this is enough for your needs or not.
>
> So long as the second active port isn't touched by Open MPI, it
> sounds just fine.
>
> One thing, though -- You mention mvapi, which IIRC is the 1st
> Generation IB stack. Is there similar functionality with the openib
> btl (for the 2nd generation IB stack)?

It looks like we never added similar logic to the Open IB transport.
I'll pass your request on to the developer of our Open IB transport.
Given our timeframe for releasing Open MPI 1.0.2, it's doubtful any
change will make that release. But it should definitely be possible
to add such functionality in a future release.

Brian