Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] System V Shared Memory for Open MPI: Request forCommunity Input and Testing
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-06-10 08:06:18


On Jun 10, 2010, at 4:57 AM, Sylvain Jeaugey wrote:

> > As for why mmap is slower. When the file is on a real (not tmpfs or other
> > ramdisk) I am 95% certain that this is an artifact of the Linux swapper/pager
> > behavior which is thinking it is being smart by "swapping ahead". Even when
> > there is no memory pressure that requires swapping, Linux starts queuing swap
> > I/O for pages to keep the number of "clean" pages up when possible. This
> > results in pages of the shared memory file being written out to the actual
> > block device. Both the background I/O and the VM metadata updates contribute
> > to the lost time. I say 95% certain because I have a colleague who looked
> > into this phenomena in another setting and I am recounting what he reported
> > as clearly as I can remember, but might have misunderstood or inserted my own
> > speculation by accident. A sufficiently motivated investigator (not me)
> > could probably devise an experiment to verify this.
> Interesting. Do you think this behavior of the linux kernel would change
> if the file was unlink()ed after attach ?

Note that OMPI does unlink the mmap'ed file after attach.

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/