Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] How to run Open MPI over TCP (Ethernet)
From: Bibrak Qamar (bibrakc_at_[hidden])
Date: 2014-05-28 02:20:05


Dear Jeff,

Thanks for the information and helping me out. I too delayed replying, I
wanted to test this but the cluster here is down. I will check it and let
you know in case it doesn't work.

Thanks
Bibrak Qamar

On Sat, May 24, 2014 at 5:23 AM, Jeff Squyres (jsquyres) <jsquyres_at_[hidden]
> wrote:

> I am sorry for the delay in replying; this week got a bit crazy on me.
>
> I'm guessing that Open MPI is striping across both your eth0 and ib0
> interfaces.
>
> You can limit which interfaces it uses with the btl_tcp_if_include MCA
> param. For example:
>
> # Just use eth0
> mpirun --mca btl tcp,sm,self --mca btl_tcp_if_include eth0 ...
>
> # Just use ib0
> mpirun --mca btl tcp,sm,self --mca btl_tcp_if_include ib0 ...
>
> Note that IPoIB is nowhere near as efficient as native verbs, so you won't
> get nearly as good performance as you do with OMPI's openib transport.
>
> Note, too, that I specifically included "--mca btl tcp,sm,self" in the
> above examples to force the use of the TCP MPI transport. Otherwise, OMPI
> may well automatically choose the native IB (openib) transport. I see you
> mentioned this in your first mail, too, but I am listing it here just to be
> specific/pedantic.
>
>
>
> On May 22, 2014, at 3:30 AM, Bibrak Qamar <bibrakc_at_[hidden]> wrote:
>
> > Hi,
> >
> > I am facing problem in running Open MPI using TCP (on 1G Ethernet). In
> practice the bandwidth must not exceed 1000 Mbps but for some data points
> (for point-to-point ping pong) it exceeds this limit. I checked with MPICH
> it works as desired.
> >
> > Following is the command I issue to run my program on TCP. Am I missing
> something?
> >
> > -bash-3.2$ mpirun -np 2 -machinefile machines -N 1 --mca btl tcp,self
> ./bandwidth.ompi
> >
> --------------------------------------------------------------------------
> > The following command line options and corresponding MCA parameter have
> > been deprecated and replaced as follows:
> >
> > Command line options:
> > Deprecated: --npernode, -npernode
> > Replacement: --map-by ppr:N:node
> >
> > Equivalent MCA parameter:
> > Deprecated: rmaps_base_n_pernode, rmaps_ppr_n_pernode
> > Replacement: rmaps_base_mapping_policy=ppr:N:node
> >
> > The deprecated forms *will* disappear in a future version of Open MPI.
> > Please update to the new syntax.
> >
> --------------------------------------------------------------------------
> > Hello, world. I am 1 on compute-0-16.local
> > Hello, world. I am 0 on compute-0-15.local
> > 1 25.66 0.30
> > 2 25.54 0.60
> > 4 25.34 1.20
> > 8 25.27 2.42
> > 16 25.24 4.84
> > 32 25.49 9.58
> > 64 26.44 18.47
> > 128 26.85 36.37
> > 256 29.43 66.37
> > 512 36.02 108.44
> > 1024 42.03 185.86
> > 2048 194.30 80.42
> > 4096 255.21 122.45
> > 8192 258.85 241.45
> > 16384 307.96 405.90
> > 32768 422.78 591.32
> > 65536 790.11 632.83
> > 131072 1054.08 948.70
> > 262144 1618.20 1235.94
> > 524288 3126.65 1279.33
> >
> > -Bibrak
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> --
> Jeff Squyres
> jsquyres_at_[hidden]
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>