Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] can't use "--mca btl_openib_cpc_include xoob"
From: Shamis, Pavel (shamisp_at_[hidden])
Date: 2011-01-13 09:11:17


Please see the comments inline .

When I use cmd "mpirun -host LB270210,CB060106,CB060107 -np 3 --mca btl openib,self,sm a_user.out" or "mpirun -host LB270210,CB060106,CB060107 -np 3 --mca btl openib,self,sm --mca btl_openib_cpc_include rdmacm a_user.out", will print correct result, but use cmd "mpirun -host LB270210,CB060106,CB060107 -np 3 --mca btl openib a_user.out", will print error mesg.

I want to know:
1) When I want to use XRC, I must have a special IB switch?

You don't have use special ib switches.

2) How can I use XRC in ompi,and on which situation the XRC feature will bring benifit?

About XRC benefit you may check this paper : http://www.open-mpi.org/papers/euro-pvmmpi-2008-xrc/

How to use.
By default openib btl use follow QPs configuration - P,128,256,192,128:S,2048,256,128,32:S,12288,256,128,32:S,65536,256,128,32
You may find some description about this format, here - https://svn.open-mpi.org/trac/ompi/ticket/1260
But , bottom line , if you want to switch default QP configuration from RC qps to XRC qps you may use follow command line parameter:
--mca btl_openib_receive_queues X,128,256,192,128 X,2048,256,128,32 X,12288,256,128,32 X,65536,256,128,32

3) If this is only way to using XRC that using "-mca btl_openib_cpc_include xoob"?
Actually this parameter does not enable XRC. Please see #2.
4) When using xoob, which transport and network protocol ompi will use?
If you will use the parameter from #2 , the xoob cpc will be used automatically.

I hope it helps.

Pavel (Pasha) Shamis

---
Application Performance Tools Group
Computer Science and Math Division
Oak Ridge National Laboratory