Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Question about '--mca btl tcp,self'
From: Jianyu Liu (jerry_leo_at_[hidden])
Date: 2014-03-15 01:18:24


>On Mar 14, 2014, at 10:16:34 AM,Jeff Squyres <jsquyres_at_[hidden]> wrote:
>
>>On Mar 14, 2014, at 10:11 AM, Ralph Castain <rhc_at_[hidden]> wrote:
>>
>>> 1. If specified '--mca btl tcp,self', which interface application will run on, use GigE adaper OR use the OpenFabrics interface in IP over IB mode (just like a high performance GigE adapter) ?
>>
>> Both - ip over ib looks just like an Ethernet adaptor
>
>
>To be clear: the TCP BTL will use all TCP interfaces (regardless of underlying physical transport). Your GigE adapter and your IP adapter both present IP interfaces to>the OS, and both support TCP. So the TCP BTL will use them, because it just sees the TCP/IP interfaces.

Thanks for your kindly input.

Please see if I have understood correctly

Assume there are two nework
   Gigabit Ethernet

     eth0-renamed : 192.168.[1-22].[1-14] / 255.255.192.0

   InfiniBand network

     ib0 : 172.20.[1-22].[1-4] / 255.255.0.0
 
    
1. If specified '--mca btl tcp,self

     The control information ( such as setup and teardown ) are routed to and passed by Gigabit Ethernet in TCP/IP mode
     The MPI messages are routed to and passed by InfiniBand network in IP over IB mode
     On the same machine, the TCP lookback device will be used for passing control and MPI messages

2. If specified '--mca btl tcp,self --mca btl_tcp_if_include ib0'

     Both of control information ( such as setup and teardown ) and MPI messages are routed to and passed by InfiniBand network in IP over IB mode
     On the same machine, The TCP lookback device will be used for passing control and MPI messages

 
3. If specified '--mca btl openib,self'

     The control information ( such as setup and teardown ) are routed to and passed by InfiniBand network in IP over IB mode
     The MPI messages are routed to and passed by InfiniBand network in RDMA mode
     On the same machine, the TCP lookback device will be used for passing control and MPI messages

4. If without specifiying any 'mca btl' parameters

     The control information ( such as setup and teardown ) are routed to and passed by Gigabit Ethernet in TCP/IP mode
     The MPI messages are routed and passed by InfiniBand network in RDMA mode
     On the same machine, the shared memory (sm) BTL will be used for control and MPI passing messages

Appreciating your kindly input

Jianyu