Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] How to time data transfers?
From: Eugene Loh (eugene.loh_at_[hidden])
Date: 2010-10-13 14:23:03


Ed Peddycoart wrote:

I need to do some performance tests on my mpi app.  I simply want to determine how long it takes for my sends from one process to be received by another process. 


Here is the code I used as my example for non-blocking send/receive...

 if( myrank == 0 ) {
  /* Post a receive, send a message, then wait */
  MPI_Irecv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &request );
  MPI_Isend( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD );
  MPI_Wait( &request, &status );
 }
 else if( myrank == 1 ) {
  /* Post a receive, send a message, then wait */
  MPI_Irecv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &request );  
  MPI_Isend( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD );
  MPI_Wait( &request, &status );
 }

First of all, you should also complete the Isend request.  One option is to turn it into a blocking Send.  Another option is to add a request to the Isend call (which is required by the API) and then turn the Wait call into a Waitall call on both requests.

I originally thought to just put a timer call before affer the rank=0 receive, but that doesn't seem to capture the complete time... see the following code.

 if( myrank == 0 ) {

  timer.start();
  /* Post a receive, send a message, then wait */
  MPI_Irecv( b, 100, MPI_DOUBLE, 1, 19, MPI_COMM_WORLD, &request );
  MPI_Isend( a, 100, MPI_DOUBLE, 1, 17, MPI_COMM_WORLD );
  MPI_Wait( &request, &status );
  timer.stop();
  elapsedTime = getElapsedTime();
 }
 else if( myrank == 1 ) {
  /* Post a receive, send a message, then wait */
  MPI_Irecv( b, 100, MPI_DOUBLE, 0, 17, MPI_COMM_WORLD, &request );  
  MPI_Isend( a, 100, MPI_DOUBLE, 0, 19, MPI_COMM_WORLD );
  MPI_Wait( &request, &status );
 }

That should work once the code is corrected.  Can you use MPI_Wtime()?  (Not necessarily a big deal, but should be a portable way of getting high-quality timings in MPI programs.)  In what sense does it not capture the complete time?

How do others time this process?  Should I send a msg from one app to the other to initiate timing, send the data I want to time?

It's common to ping-pong many times back and forth.  There may be one or more "warm-up" iterations (to make sure both processes are ready and all resources used have been touched and warmed/woken up) and other iterations to check reproducibility of results.  Also, one might have many iterations between the timer calls to amortize the overhead of the timer call.


 if( myrank == 0 ) {

  MPI_Irecv( b, 100, MPI_DOUBLE, 1, startTimeTag, MPI_COMM_WORLD, &request );
  MPI_Wait( &request, &status );
  timer.start();
  MPI_Irecv( b, 100, MPI_DOUBLE, 1, dataTag, MPI_COMM_WORLD, &request );
  MPI_Wait( &request, &status );
  timer.stop();
  elapsedTime = getElapsedTime();
 }
 else if( myrank == 1 ) {

  MPI_Isend( a, 100, MPI_DOUBLE, 0, startTimerTag, MPI_COMM_WORLD );
  MPI_Wait( &request, &status );
  MPI_Isend( b, 100, MPI_DOUBLE, 0, dataTag , MPI_COMM_WORLD, &request );  
  MPI_Wait( &request, &status );
 }

Ed


_______________________________________________ users mailing list users@open-mpi.org http://www.open-mpi.org/mailman/listinfo.cgi/users