Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2006-01-26 08:47:24


This looks like a problem with the memory allocator. It could be a
genuine problem with Open MPI, or it could be a memory fault in your
application (that happens to dead-end in one of our libraries because
we intercept memory allocation functions). From the stack trace, it
looks like you're in the middle of a complex deallocation of some C++
objects, so I really can't tell (i.e., not in an MPI function at all).

Can you do a few things:

- configure your Open MPI --with-memory-manager=none and see if the
problem goes away. This tells Open MPI to not intercept memory
manager functions, so if you still have the problem, it's more likely
to be a problem in your application than in OMPI.

- run your application through a memory-checking debugger (such as
valgrind) and see if it identifies any memory faults within your code.

- send the additional information for run-time problems listed on
http://www.open-mpi.org/community/help/

Thanks!

On Jan 25, 2006, at 10:39 PM, Glenn Morris wrote:

>
> I tried nightly snapshot 1.1a1r8803 and it said the following. I'm
> willing to try and debug this further, but would need some guidance. I
> have access to totalview.
>
>
> Signal:11 info.si_errno:0(Success) si_code:2(SEGV_ACCERR)
> Failing at addr:0x97421004
> [0] func:/afs/slac.stanford.edu/g/ki/users/gmorris/tmp/
> ompi-1.1a1r8803/lib/libopal.so.0 [0x1cc9fa]
> [1] func:/lib/tls/libpthread.so.0 [0xfd2f80]
> [2] func:/afs/slac.stanford.edu/g/ki/users/gmorris/tmp/
> ompi-1.1a1r8803/lib/libopal.so.0(free+0x5e) [0x1cf0a2]
> [3] func:./cosmomc(for_deallocate+0x56) [0x80d8806]
> [4] func:./cosmomc(for_dealloc_allocatable+0x59) [0x80d886d]
> [5] func:./cosmomc(spline_+0x4f2) [0x805a2ea]
> [6] func:./cosmomc(cambmain_mp_initsourceinterpolation_+0x243)
> [0x8089b65]
> [7] func:/afs/slac.stanford.edu/package/intel_tools/compiler8.1/
> i386_linux24/ifort/lib/libguide.so(__kmp_invoke_microtask+0x4d)
> [0x19f8cd]
> [8] func:/afs/slac.stanford.edu/package/intel_tools/compiler8.1/
> i386_linux24/ifort/lib/libguide.so(__kmpc_invoke_task_func+0xa2)
> [0x18fea6]
> [9] func:/afs/slac.stanford.edu/package/intel_tools/compiler8.1/
> i386_linux24/ifort/lib/libguide.so(__kmp_internal_fork+0x19b)
> [0x1900a1]
> [10] func:/afs/slac.stanford.edu/package/intel_tools/compiler8.1/
> i386_linux24/ifort/lib/libguide.so(__kmp_fork_call+0x334) [0x18af18]
> [11] func:/afs/slac.stanford.edu/package/intel_tools/compiler8.1/
> i386_linux24/ifort/lib/libguide.so(__kmpc_fork_call+0x35) [0x19242d]
> [12] func:./cosmomc(cambmain_mp_initsourceinterpolation_+0xa8)
> [0x80899ca]
> [13] func:./cosmomc(cambmain_mp_cmbmain_+0x8a8) [0x8085c34]
> [14] func:./cosmomc(camb_mp_camb_getresults_+0x99) [0x80936db]
> [15] func:./cosmomc(camb_mp_camb_gettransfers_+0x117) [0x8093387]
> [16] func:./cosmomc(cmb_cls_mp_getcls_+0x102) [0x80aa82a]
> [17] func:./cosmomc(calclike_mp_getloglikepost_+0x1f9) [0x80b43e9]
> [18] func:./cosmomc(calclike_mp_getloglike_+0x23e) [0x80b41d0]
> [19] func:./cosmomc(montecarlo_mp_mcmcsample_+0x130) [0x80b68dc]
> [20] func:./cosmomc(MAIN__+0x15a3) [0x80b885b]
> [21] func:./cosmomc(main+0x20) [0x8059758]
> [22] func:/lib/tls/libc.so.6(__libc_start_main+0xda) [0x23c79a]
> [23] func:./cosmomc(sinh+0x49) [0x8059611]
> *** End of error message ***
> 1 process killed (possibly by Open MPI)
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/