Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-10-24 20:01:44

WHAT: Add a "connect" field to the openib BTL INI file to specify
which connect pseudo-component(s) (CPCs) can be used for wireup of a
given device.

WHY: 2 new CPCs are being written (by different parties); certain
devices can only use certain CPCs.

WHERE: Mainly in btl_openib_ini.*, but probably also touch a few
other btl_openib_* files.

WHEN: Sometime in the near future; no specific time set.

TIMEOUT: Friday, 2 Nov 2007


In addition to the already-existing "oob" connect pseudo-component
(CPC), two more are in the works:

- XRC: Pasha/Mellanox is working on a new XRC CPC for ConnectX HCAs.

- RDMA CM: Welcome to Jon Mason from Open Grid Computing (for
Chelsio) who will be doing iWARP-related work in the udapl and/or
openib BTLs. Either he or I will be implementing the RDMA CM-based CPC.

My proposal is that the "connect" field can be added to the INI file
and take a comma-delimited list of values of acceptable CPCs for a
given device. For example, the ConnectX HCA can take the following

     connect = xrc, rdma_cm, oob


- first, try the XRC CPC to try to make the connection
- if that fails, try the RDMA CM CPC
- if that fails, try the OOB CPC
- if that fails, fail the connection

iWARP-based NICs can use the following value:

     connect = rdma_cm

If no "connect" value is specified, then default value of "oob,
rdma_cm" can be assumed (possibly someday changing to "rdma_cm, oob").

I mention this here on the devel list because disparate groups are
working on different CPC's; coordination will be required to
implement this arbitration mechanism.


Jeff Squyres
Cisco Systems