Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Profiling OpenMPI routines
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-10-01 22:17:27


Aniruddha Marathe wrote:

>I am trying to profile (get the call graph/call sequence of) Open MPI
>communication routines using GNU Profiler (gprof) since the
>communication calls are implemented using macros and it's harder to
>trace them statically. In order to do that I compiled the OpenMPI
>source code with following options supplied to 'configure' tool:
>
>./configure CFLAGS=-pg CPPFLAGS=-pg --enable-debug
>--prefix=/home/amarathe/mpi/svn_openmpi/install
>
>When I recompiled my test MPI application that does MPI_Send and
>MPI_Recv with the new library, it generated gmon.out file as expected
>(I ran it as 'mpirun -np 2 send_recv'). However, running 'gprof' on
>this file didn't provide any information such as the call graphs for
>MPI_Send or MPI_Recv. Following is the only function call that I see
>in the output:
>
>$ gprof send_recv gmon.out
>...
>...
>% cumulative self self total
>time seconds seconds calls Ts/call Ts/call name
>0.00 0.00 0.00 25 0.00 0.00 data_start
>...
>...
>
>I would like to know if anyone has done something similar with gprof
>or any other open source tool with OpenMPI code.
>
>(I found a similar, fairly recent post on the mailing list, but it
>seems to talk about profiling the MPI application itself and not the
>OpenMPI library routines -
>http://www.open-mpi.org/community/lists/users/2009/04/8999.php)
>
>
Open source tool or free download? That is, do you really need to be
able to see the tool's source code, or are you just interested in
avoiding license fees? In any case, since that post you mention, a FAQ
has appeared on performance tools. Check
http://www.open-mpi.org/faq/?category=perftools

You make an important distinction between profiling MPI applications
versus profiling the library itself, and many tools will help just with
applications. But I've used Sun Studio for profiling Open MPI.
Ideally, you should ./configure with -g among the compilation switches
so that you get symbolic information about the library, but that isn't
necessary. The use of macros and dynamically loaded objects makes
correlating profiles with source code hard, but it works. When you
bring the Analyzer up, I think you also have to unhide the symbols
within the MPI library, which as I remember are hidden by default.
Anyhow, it works and I've learned a lot doing things this way.