Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] btl_openib_cpc_include rdmacm questions
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2011-04-28 08:07:03

On Apr 27, 2011, at 10:02 AM, Brock Palen wrote:

> Argh, our messed up environment with three generations on infiniband bit us,
> Setting openib_cpc_include to rdmacm causes ib to not be used on our old DDR ib on some of our hosts. Note that jobs will never run across our old DDR ib and our new QDR stuff where rdmacm does work.

Hmm -- odd. I use RDMACM on some old DDR (and SDR!) IB hardware and it seems to work fine.

Do you have any indication as to why OMPI is refusing to use rdmacm on your older hardware, other than "No OF connection schemes reported..."? Try running with --mca btl_base_verbose 100 (beware: it will be a truckload of output). Make sure that you have rdmacm support available on those machines, both in OMPI and in OFED/the OS.

> I am doing some testing with:
> export OMPI_MCA_btl_openib_cpc_include=rdmacm,oob,xoob
> What I want to know is there a way to tell mpirun to 'dump all resolved mca settings' Or something similar.

I'm not quite sure what you're asking here -- do you want to override MCA params on specific hosts?

Jeff Squyres
For corporate legal information go to: