Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: [OMPI devel] ptmalloc and pin down cache problems again
From: Gleb Natapov (glebn_at_[hidden])
Date: 2008-01-07 10:14:08

Hi Brian,

 I encountered problem with ptmalloc an registration cache. I see that
you (I think it was you) disabled shrinking of a heap memory allocated
by sbrk by setting MORECORE_CANNOT_TRIM to 1. The comment explains that
it should be done because freeing of small objects is not reentrant so
if ompi memory subsystem callback will call free() the code will deadlock.
And the trick indeed works in single threaded programs, but in multithreaded
programs ptmalloc may allocate a heap not only by sbrk, but by mmap too. This
is called "arena". Each thread may have arenas of its own. The problem is
that ptmalloc may free an arena by calling munmap() and then free() that
is called from our callback deadlocks. I tried to compile with USE_ARENAS set
to 0, but the code doesn't compile. I can fix compilation problem of
cause, but it seems that it is not so good idea to disable this feature.
The ptmalloc scalability depends on it (and even if we will disable it
ptmalloc may still create arena by mmap if sbrk fails). I see only one
way to solve this problem: to not call free() inside mpool callbacks.
If freeing of a memory is needed (and it is needed since IB unregister
calls free()) the works should be deferred. For IB mpool we can check what
needs to be unregistered inside a callback, but actually call unregister()
from next mpool->register() call. Do you see any problems with this