Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-02-06 10:43:15

On Feb 2, 2007, at 11:22 AM, Alex Tumanov wrote:

> That really did fix it, George:
> # mpirun --prefix $MPIHOME -hostfile ~/testdir/hosts --mca btl
> tcp,self --mca btl_tcp_if_exclude ib0,ib1 ~/testdir/hello
> Hello from Alex' MPI test program
> Process 0 on out of 2
> Hello from Alex' MPI test program
> Process 1 on compute-0-0.local out of 2
> It never occurred to me that the headnode would try to communicate
> with the slave using infiniband interfaces... Orthogonally, what are

The problem here is that since your IB IP addresses are
"public" (meaning that they're not in the IETF defined ranges for
private IP addresses), Open MPI assumes that they can be used to
communicate with your back-end nodes on the IPoIB network. See this
FAQ entry for details:

If you update your IP addresses to be in the private range, Open MPI
should do the Right routability computations and you shouldn't need
to exclude anything.

> the industry standard OpenMPI benchmark tests I could run to perform a
> real test?

Just about anything will work -- NetPIPE, the Intel Benchmarks, ...etc.

Jeff Squyres
Server Virtualization Business Unit
Cisco Systems