Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] Shared Memory RDMA and eager messages
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-05-15 07:15:30

On May 15, 2008, at 5:55 AM, Torje Henriksen wrote:

> 1. In the BTL SM component (both in "ompi_info --param btl sm" and the
> source), there is mention of RDMA. Is RDMA used in ths component? Is
> any special hardware needed for using it?

No and no. The SM BTL uses a copyout/copyin mechanism (copy out to
shared memory, copy in to the target process) -- it does not directly
copy from one process to another.

The OB1 PML uses "RDMA-like" semantics; I'm guessing that's what it's
referring to.

> 2. As for eager vs non-eager messages. Is the only difference between
> these the amount of memory allocated for a fragment? Is there any
> rendevouz protocol used for large messages or anything like that?

The rendesvouz protocol is enforced by the upper level PML (OB1) -- it
is basically the lower-layer-independent "protocol engine". OB1
decides the exact protocol to use (short, long, pipelined, etc.), and
the lower later BTLs decide how to get the bits there.

> The module based architecture is great, but sometimes I find it hard
> to figure out what is happening :)

Indeed. This is definitely a tradeoff when you only see small pieces
of the puzzle at a time. :-\

Jeff Squyres
Cisco Systems