Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Multi-rail support
From: Rolf Vandevaart (Rolf.Vandevaart_at_[hidden])
Date: 2010-03-25 10:31:42


You need to ensure your cluster has some type of TCP connectivity. That
can be via a non-IB interface if you have that, or configure one of your
IB interfaces with IP. It is needed so the run-time environment can
start the job and exchange endpoint information to all the processes.
You do not need to configure both IB ports or do IB bonding.
Rolf

On 03/25/10 09:41, PN wrote:
> A quick question. Do I need to configure different IP for both IB ports
> before running mpirun?
> Or configure an IP and bond both IB ports? Or simply configure one IP
> for ib0 is enough?
> Thanks a lot.
>
> PN
>
>
> 2010/3/25 Rolf Vandevaart <Rolf.Vandevaart_at_[hidden]
> <mailto:Rolf.Vandevaart_at_[hidden]>>
>
> They will automatically be used by the library. There is nothing
> special that you need to do. Unfortunately, there is no simple way
> to tell if they are being used. I would suggest that you
> specifically call them out in different calls to mpirun to make sure
> they are both working. If they both work individually, then you can
> assume they are both being used. You will only see better
> performance with larger messages. Something like this:
>
> mpirun -mca btl_openib_if_include mlx4_0 a.out
> mpirun -mca btl_openib_if_include mlx4_1 a.out
>
> You need to find out your device names from the command ibv_devinfo
> or ibv_devices.
>
> Rolf
>
>
> On 03/25/10 05:53, Dmitry Kuzmin wrote:
>
> Hi there,
>
> We have a cluster with 2 HCAs installed on each node (Mellanox
> ConnectX IB QDR cards). It's not clear from the documentation
> how we could use both of them for MPI communications.
> What should we do to enable 2 cards and how we can check that
> both are using?
>
> Thanks in advance!
> Dmitry
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> users mailing list
> users_at_[hidden] <mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
> --
>
> =========================
> rolf.vandevaart_at_[hidden] <mailto:rolf.vandevaart_at_[hidden]>
> 781-442-3043
> =========================
> _______________________________________________
> users mailing list
> users_at_[hidden] <mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
>
> --
> Best Regards,
> PN Lai
> Storage Manager / HPC Specialist
> Galactic Computng Corp.
> Tel: 86-755-26733939 ext 826
> Mobile: 86-13823161729 / 852-60333151
> Fax: 86-755-26733780
> URL: http://www.galactic.com.hk
>
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
=========================
rolf.vandevaart_at_[hidden]
781-442-3043
=========================