Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-10-25 17:39:07

On Oct 25, 2007, at 11:18 AM, Gleb Natapov wrote:

>>> I don't think xrc should be used by default even if HW supports it.
>>> Only if
>>> special config option is set xrc should be attempted.
>> Why?
> XRC is a crippled RC protocol for scalability sake. Its use makes
> progress of one process depend on behaviour of other processes on the
> same node which make cause different interesting effects. And of cause
> SW flow control is not possible when using XRC, so for small jobs it
> will be actually slower. I don't thinks it is wise to use XRC over
> regular RC if there is a choice.

Fun stuff. :-)

>>> And xrc availability
>>> can be tested in runtime without additional options in ini file.
>> Is there a flag on the device / port that indicates XRC availability?
> XRC requires creation of special kind of QP. If this fails XRC is not
> available.

That's awkward, but ok. :-)

>> So do you want the arbitration rules for which CPC to be used to be
>> hard-coded in the openib component (possibly overridden by MCA
>> parameter to force a specific selection)?
> Not hard-coded, but controlled by regular mca mechanism, with default
> behaviour dependant on HCA type. Not something new. We have this with
> SRQ.

Something like this?

if (mca_specified)
   setup to use MCA-specified CPC
else if (iwarp_device)
   setup to use rdma_cm CPC
   setup to use oob CPC

Jeff Squyres
Cisco Systems