Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Swamy Kandadai (swamy_at_[hidden])
Date: 2006-06-23 22:22:01


Dr. Swamy N. Kandadai
Certified Sr. Consulting IT Specialist
HPC Benchmark Center
System & Technology Group, Poughkeepsie, NY
Phone:( 845) 433 -8429 (8-293) Fax:(845)432-9789
swamy_at_[hidden]
http://w3.ibm.com/sales/systems/benchmarks

----- Forwarded by Swamy Kandadai/Poughkeepsie/IBM on 06/23/2006 10:21 PM
-----
                                                                           
             Swamy
             Kandadai/Poughkee
             psie/IBM To
                                       users_at_[hidden]
             06/23/2006 09:52 cc
             PM
                                                                   Subject
                                       OpenMPI version 1.1
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           
                                                                           

Hi:

I am trying to run OpenMPI on a couple of nodes. These nodes have several
interfaces: eth0 (which is a GigE),
eth1 (which is a GigE with Jumbo frames enabled), IpoIB, myr0 in addition
to loopback (l0).

I want to use exclusively eth1 and I am running with this option:

mpirun --mca btl_tcp_if_include eth1 -machinefile hf -np 2 IMB-MPI1

where IMB-MPI1 is the Intel message passing benchmark.

Different times, it has different behaviors:

I ran on one set of nodes and I got a typical GigE behavior (around 100
MB/s). On a different pairs of nodes
it is giving me the BW consistent with IpoIB (around 700 MB/s).

Can u help me what I am doing wrong? How can I force it to use eth1 on all
nodes?

I just built OpenMPI with the following option:

./configure --prefix=$BINDIR --disable-io-romio

Thanks
Swamy

Dr. Swamy N. Kandadai
Certified Sr. Consulting IT Specialist
HPC Benchmark Center
System & Technology Group, Poughkeepsie, NY
Phone:( 845) 433 -8429 (8-293) Fax:(845)432-9789
swamy_at_[hidden]
http://w3.ibm.com/sales/systems/benchmarks