Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] Memory corruption with mpool
From: Christopher Yeoh (cyeoh_at_[hidden])
Date: 2009-11-02 08:02:10


Hi Mondrian,

On Mon, 02 Nov 2009 13:22:11 +0100
Mondrian Nuessle <nuessle_at_[hidden]> wrote:
>
> If I turn on mpi_leave_pinned (and thus the registration cache is
> actually used), I see occasional memory corruption issues for example
> when I call MPI_Allreduce often.
>
> Debugging with valgrind did not lead to any clues, since OMPI refuses
> to run in that case. If I turn off mpi_leave_pinned, everything seems
> to be fine.
>
> I tested on version 1.3.3 and 1.3.4rc1.
>
> Do you have any suggestions how to investigate this situation?

Have you got OMPI_ENABLE_DEBUG defined? The symptoms of what you are
seeing sound like what might happen if debug is off and you trigger an
issue I posted about here related to thread safety of mpool.

http://www.open-mpi.org/community/lists/devel/2009/10/7042.php

If OMPI_ENABLE_DEBUG is defined it will abort if pthread_mutex_lock
returns EDEADLK (see opal_mutex_lock), but if not, the code proceeds
without a lock which could cause memory corruption.

Chris

-- 
cyeoh_at_[hidden]