Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] --mca btl_openib_if_include
From: Mostyn Lewis (Mostyn.Lewis_at_[hidden])
Date: 2008-10-16 21:10:46


Hello,

Using today's SVN 1.4a1r19757

with
MCA='--mca btl_openib_verbose 1 --mca btl openib,self --mca btl_openib_if_include "mlx4_0:1,mlx4_1:1"'

ibstatus (OFED 1.3.1) gives:
Infiniband device 'mlx4_0' port 1 status:
         default gid: fe80:0000:0000:0000:0003:ba00:0100:71a1
         base lid: 0x2f
         sm lid: 0x1
         state: 4: ACTIVE
         phys state: 5: LinkUp
         rate: 20 Gb/sec (4X DDR)

Infiniband device 'mlx4_0' port 2 status:
         default gid: fe80:0000:0000:0000:0003:ba00:0100:71a2
         base lid: 0x0
         sm lid: 0x0
         state: 1: DOWN
         phys state: 2: Polling
         rate: 10 Gb/sec (4X)

Infiniband device 'mlx4_1' port 1 status:
         default gid: fe80:0000:0000:0000:0003:ba00:0100:70b9
         base lid: 0x30
         sm lid: 0x1
         state: 4: ACTIVE
         phys state: 5: LinkUp
         rate: 20 Gb/sec (4X DDR)

Infiniband device 'mlx4_1' port 2 status:
         default gid: fe80:0000:0000:0000:0003:ba00:0100:70ba
         base lid: 0x0
         sm lid: 0x0
         state: 1: DOWN
         phys state: 2: Polling
         rate: 10 Gb/sec (4X)

OpenMPI says for a:
mpirun --prefix /tools/openmpi/1.4a1r19757_svn/connectx/gcc64/4.1.2/openib/rh_EL_4/x86_64/xeon -x LD_LIBRARY_PATH --mca btl_openib_verbose 1 --mca btl openib,self --mca btl_openib_if_include "mlx4_0:1,mlx4_1:1" -np 4 -machinefile dhosts ./IMB-MPI1.openmpi

--------------------------------------------------------------------------
WARNING: One or more nonexistent OpenFabrics devices/ports were
specified:

   Host: r4450_3
   MCA parameter: mca_btl_if_include
   Nonexistent entities: "mlx4_0:1,mlx4_1:1"

These entities will be ignored. You can disable this warning by
setting the btl_openib_warn_nonexistent_if MCA parameter to 0.
--------------------------------------------------------------------------

Scali 5.6 works in dual-rail mode with this as does mvapich2-1.2rc2.

What am I doing wrong, please?

DM