Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-06-10 08:05:32

On Jun 10, 2010, at 4:43 AM, Paul H. Hargrove wrote:

> One should not ignore the option of POSIX shared memory: shm_open() and
> shm_unlink(). When present this mechanism usually does not suffer from
> the small (eg 32MB) limits of SysV, and uses a "filename" (in an
> abstract namespace) which can portably be up 14 characters in length.
> Because shm_unlink() may be called as soon as the final process has done
> its shm_open() one can get approximately the safety of the IPC_RMID
> mechanism, but w/o being restricted to Linux.

FWIW, with the infrastructure work that Sam did, it would probably be the work of about an hour or two to add shm_open()/etc. into the common sm stuff.

> I have used POSIX shared memory for another project and found it works
> well on Linux, Solaris (10 and Open), FreeBSD and AIX. That is probably
> a narrow coverage than SysV, but still worth consideration IMHO. With
> mmap(), SysV and POSIX (plus XPMEM on the SGI Altix) as mechanisms for
> sharing memory between processes, I think we have an argument for a
> full-blown "shared pages" framework as opposed to just a "mpi_common_sm"
> MCA parameter. That brings all the benefits like possibly "failing
> over" from one component to another (otherwise less desired) one if some
> limit is exceeded. For instance, SysV could (for a given set of
> priorities) be used by default, but mmap-on-real-fs could be
> automatically selected when the requested/required size exceeds the
> shmmax value.

That's more-or-less what Sam did.

Sam -- if the shmat stuff fails because the limits are too low, it'll (silently) fall back to the mmap module, right?

Jeff Squyres
For corporate legal information go to: