Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Memory manager
From: Gleb Natapov (glebn_at_[hidden])
Date: 2008-05-20 01:53:52


On Tue, May 20, 2008 at 12:17:02PM +1000, Terry Frankcombe wrote:
> To tell you all what noone wanted to tell me, yes, it does seem to be
> the memory manager. Compiling everything with
> --with-memory-manager=none returns the vmem use to the more reasonable
> ~100MB per process (down from >8GB).
>
> I take it this may affect my peak bandwidth over infiniband. What's the
> general feeling about how bad this is?
You will not be able to use "-mca mpi_leave_pinned 1" parameter and your
micro benchmark performance will be bad. Real application will see the
difference only if it reuses communication buffers frequently.

>
>
> On Tue, 2008-05-13 at 13:12 +1000, Terry Frankcombe wrote:
> > Hi folks
> >
> > I'm trying to run an MPI app on an infiniband cluster with OpenMPI
> > 1.2.6.
> >
> > When run on a single node, this app is grabbing large chunks of memory
> > (total per process ~8.5GB, including strace showing a single 4GB grab)
> > but not using it. The resident memory use is ~40MB per process. When
> > this app is compiled in serial mode (with conditionals to remove the MPI
> > calls) the memory use is more like what you'd expect, 40MB res and
> > ~100MB vmem.
> >
> > Now I didn't write it so I'm not sure what extra stuff the MPI version
> > does, and we haven't tracked down the large memory grabs.
> >
> > Could it be that this vmem is being grabbed by the OpenMPI memory
> > manager rather than directly by the app?
> >
> > Ciao
> > Terry
> >
> >
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

--
			Gleb.