Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Troy Telford (ttelford_at_[hidden])
Date: 2006-03-10 02:24:50


On Mar 9, 2006, at 9:18 PM, Brian Barrett wrote:

> On Mar 9, 2006, at 6:41 PM, Troy Telford wrote:
>
>> I've got a machine that has the following config:
>>
>> Each node has two InfiniBand ports:
>> * The first port is on fabric 'a' with switches for 'a'
>> * The second port is on fabric 'b' with separate switches for 'b'
>> * The two fabrics are not shared ('a' and 'b' can't communicate
>> with one
>> another)
>>
>> I believe that Open MPI is perfectly capable of stripeing over both
>> fabric
>> 'a' and 'b', and IIRC, this is the default behavior.
>>
>> Does Open MPI handle the case where Open MPI puts all of its
>> traffic on
>> the first IB port (ie. fabric 'a'), and leaves the second IB port
>> (ie.
>> fabric 'b') free for other uses (I'll use NFS as a humorous example).
>>
>> If so, is there any magic required to configure it thusly?
>
> With mvapi, we don't have the functionality in place for the user to
> specify which HCA port is used. The user can say that at most N HCA
> ports should be used through the btl_mvapi_max_btls MCA parameter.
> So in your case, if you ran Open MPI with:
>
> mpirun -mca btl_mvapi_max_btls 1 -np X ./foobar
>
> Only the first active port would be used for mvapi communication.
> I'm not sure if this is enough for your needs or not.

So long as the second active port isn't touched by Open MPI, it
sounds just fine.

One thing, though -- You mention mvapi, which IIRC is the 1st
Generation IB stack. Is there similar functionality with the openib
btl (for the 2nd generation IB stack)?

Thanks!

Troy
  Telford