Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: make mpi_leave_pinned=1 the default
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-07-07 15:10:12


On Jul 6, 2008, at 1:28 PM, Patrick Geoffray wrote:

>> WHAT: make mpi_leave_pinned=1 by default when a BTL is used that
>> would benefit from it (when possible; 0 when not, obviously)
>
> The probable reason registration cache (aka leave_pinned) is
> disabled by default is that it may be unsafe. Even if you use
> mallocopt to never return memory to the OS, how do you guarantee that:
> * malloc always enforce the mallocopt *hints*.
> * pinned memory can safely be fork()ed (system() for example).
> * pinned memory can safely be unmmap()ed (Direct I/O or file mapping
> for example).
>
> If you can't, one solution may be to write a simple MPI code that
> corrupts MVAPICH and make some noise about it.

I can appreciate that, but it still remains that we are the *only*
OpenFabrics MPI that doesn't do this by default. So an app would
break *all* of them. To date, there hasn't been an app that caused
anyone to change their defaults with regard to this issue (AFAIK).

- I agree that we don't know that the mallopt hint will be obeyed; I
don't know how to test for that.
- pinned memory is already unsafe in OpenFabrics when forking.
- we can catch munmap (it's in the code base already) in most cases,
so I'm not too worried about that.

-- 
Jeff Squyres
Cisco Systems