Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: changes to modex
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-04-02 10:35:03

On Apr 2, 2008, at 10:27 AM, Gleb Natapov wrote:
> In the case of openib BTL what part of modex are you going to send
> using
> proc_send() and what part using node_send()?

In the /tmp-public/openib-cpc2 branch, almost all of it will go to the
node_send(). The CPC's will likely now get 2 buffers: one for
node_send, and one for proc_send.

The ibcm CPC, for example, can do everything in node_send (the
service_id that I use in the ibcm calls is the proc's PID; ORTE may
supply peer PIDs directly -- haven't decided if that's a good idea yet
or not -- if it doesn't, the PID can be sent in the proc_send data).
The rdmacm CPC may need a proc_send for the listening TCP port number;
still need to figure that one out.

If we use carto to limit hcas/ports are used on a given host on a per-
proc basis, then we can include some proc_send data to say "this proc
only uses indexes X,Y,Z from the node data". The indexes can be
either uint8_ts, or maybe even a variable length bitmap.

Jeff Squyres
Cisco Systems