Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

From: Tim Prins (tprins_at_[hidden])
Date: 2007-08-23 13:40:25


Hi Josh,

I am not an expert in this area of the code, but I'll give it a shot.

(I assume you are using linux due to your email address) When using the memory
manager (which is the default on linux), we wrap malloc/realloc/etc with
ptmalloc2 (which is the same allocator used in glibc 2.3.x).

What I believe is happening is that ptmalloc2 is requesting more memory than
necessary from the OS, and then lazily releasing it back. Try looking in the
ompi source at opal/mca/memory/ptmalloc2/README (
https://svn.open-mpi.org/trac/ompi/browser/tags/v1.2-series/v1.2.3/opal/mca/memory/ptmalloc2/README#L121 ).

This mentions some environment variables that can be set to alter ptmalloc2's
behavior, although I have no idea if they work.

Alternatively, if you are not using a high performance network, there is
little reason to use the memory manager, so you could just disable it.

Tim

On Thursday 23 August 2007 10:18:45 am Josh Aune wrote:
> I have found that the infiniserv MPI that comes with our IB software
> distribution tracks the same behaviour as gcc (releaseing memory on
> realloc). I have also found that building openmpi with
> --without-memory-manager makes openmpi track the same behaviour as
> glibc. I'm guessing that there is a bug in the pinned pages caching
> code?
>
> On 8/21/07, Josh Aune <luken_at_[hidden]> wrote:
> > The realloc included with openmpi 1.2.3 is not releasing memory to the
> > OS and is causing apps to go into swap. Attached is a little test
> > program that shows calls to realloc not releasing the memory when
> > compiled using mpicc, but when compiled directly with gcc (or icc)
> > calling realloc() frees any memory no longer needed.
> >
> > Is this a bug?
> >
> > If not, how can I force openmpi to free the memory that the allocator
> > is sitting on?
> >
> > Thanks,
> > Josh
> >
> > Sample output. Note the delta between 'total' and 'malloc held' when
> > compiled with mpicc and how the gcc compiled versions track perfectly.
> >
> > $ mpicc -o realloc_test realloc_test.c
> > $ ./realloc_test
> > ...
> > malloc/realloc/free test
> > malloc() 50 MB, total 50 MB, malloc held 50 MB
> > realloc() 1 MB, total 1 MB, malloc held 50 MB
> > malloc() 50 MB, total 51 MB, malloc held 100 MB
> > realloc() 1 MB, total 2 MB, malloc held 100 MB
> > malloc() 50 MB, total 52 MB, malloc held 150 MB
> > realloc() 1 MB, total 3 MB, malloc held 150 MB
> > malloc() 50 MB, total 53 MB, malloc held 200 MB
> > realloc() 1 MB, total 4 MB, malloc held 200 MB
> > malloc() 50 MB, total 54 MB, malloc held 250 MB
> > realloc() 1 MB, total 5 MB, malloc held 250 MB
> > free() 1 MB, total 4 MB, malloc held 200 MB
> > free() 1 MB, total 3 MB, malloc held 150 MB
> > free() 1 MB, total 2 MB, malloc held 100 MB
> > free() 1 MB, total 1 MB, malloc held 50 MB
> > free() 1 MB, total 0 MB, malloc held 0 MB
> > ...
> >
> > $ gcc -o realloc_test realloc_test.c
> > $ ./realloc_test
> > ...
> > malloc/realloc/free test
> > malloc() 50 MB, total 50 MB, malloc held 50 MB
> > realloc() 1 MB, total 1 MB, malloc held 1 MB
> > malloc() 50 MB, total 51 MB, malloc held 51 MB
> > realloc() 1 MB, total 2 MB, malloc held 2 MB
> > malloc() 50 MB, total 52 MB, malloc held 52 MB
> > realloc() 1 MB, total 3 MB, malloc held 3 MB
> > malloc() 50 MB, total 53 MB, malloc held 53 MB
> > realloc() 1 MB, total 4 MB, malloc held 4 MB
> > malloc() 50 MB, total 54 MB, malloc held 54 MB
> > realloc() 1 MB, total 5 MB, malloc held 5 MB
> > free() 1 MB, total 4 MB, malloc held 4 MB
> > free() 1 MB, total 3 MB, malloc held 3 MB
> > free() 1 MB, total 2 MB, malloc held 2 MB
> > free() 1 MB, total 1 MB, malloc held 1 MB
> > free() 1 MB, total 0 MB, malloc held 0 MB
> > ...
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users