Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] mca_pml_ob1_rdma_btls and leave_pinned logic
From: George Bosilca (bosilca_at_[hidden])
Date: 2009-09-28 20:58:46


Roman,

Before going into explaining the logic, let me state that the memory
is registered (if required/supported) by the BTLs. However, this is
done only at the moment when the memory segment is involved in any
kind of communication.

We do not want to replicate this at the PML level, in order to make
sure that the amount of memory registered at any moment is minimal. In
other words, the PML leave the decision on when to register and when
to unregister to the BTLs. However, in order to speedup the code a
little bit (and to keep things tidy), the PML will help the BTLs to
work around the memory registration issue. And the code you pointed
out is exactly the place where we do it.

We need to register the memory if leave_pinned is TRUE, as registering
will leave a trace. If leave_pinned is FALSE then we only check if
somehow this memory is not already registered (by some BTL). In this
case, there is no need to create a registration in the PML (if
required the BTL will do it when needed).

   george.

On Sep 28, 2009, at 13:44 , Roman Cheplyaka wrote:

> Hi,
> I'm trying to dig into OpenMPI sources but have some problems. Can
> anyone explain the logic of the following code from
> mca/pml/ob1/pml_ob1_rdma.c please?
>
> if(!mca_pml_ob1.leave_pinned) {
> /* look through existing registrations */
> btl_mpool->mpool_find(btl_mpool, base, size, &reg);
> } else {
> /* register the memory */
> btl_mpool->mpool_register(btl_mpool, base, size, 0,
> &reg);
> }
>
> It seems to me that we should register new memory if leave_pinned is
> FALSE (i.e. no existing registrations avaliable) and search through
> existing registrations otherwise, but the logic is inversed here.
>
> --
> Roman I. Cheplyaka
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel