Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] memchecker overhead?
From: Jed Brown (jed_at_[hidden])
Date: 2009-10-26 16:43:12

Jeff Squyres wrote:

> Verbs and Open MPI don't have these options on by default because a)
> you need to compile against Valgrind's header files to get them to
> work, and b) there's a tiny/small amount of overhead inserted by OMPI
> telling Valgrind "this memory region is ok", but we live in an
> intensely competitive HPC environment.

It's certainly competitive, but we spend most of our implementation time
getting things correct rather than tuning. The huge speed benefits come
from algorithmic advances, and finding bugs quickly makes the
implementation of new algorithms easier. I'm not arguing that it should
be on by default, but it's helpful to have an environment where the
lower-level libs are valgrind-clean. These days, I usually revert to
MPICH when hunting something with valgrind, but use OMPI most other

> The option to enable this Valgrind Goodness in OMPI is --with-valgrind.
> I *think* the option may be the same for libibverbs, but I don't
> remember offhand.

I see plenty of warning over btl sm. Several variations, including the

--enable-debug --enable-mem-debug --enable-mem-profile \
   --enable-memchecker --with-valgrind=/usr

were not sufficient. (I think everything in this line except
--with-valgrind increases the number of warnings, but it's nontrivial
with plain --with-valgrind.)