Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Do MPI calls ever sleep?
From: Douglas Guptill (douglas.guptill_at_[hidden])
Date: 2010-07-21 14:57:23


Hi David:

On Wed, Jul 21, 2010 at 02:10:53PM -0400, David Ronis wrote:
> I've got a mpi program on an 8-core box that runs in a master-slave
> mode. The slaves calculate something, pass data to the master, and
> then call MPI_Bcast waiting for the master to update and return some
> data via a MPI_Bcast originating on the master.
>
> One of the things the master does while the slaves are waiting is to
> make heavy use of fftw3 FFT routines which can support multi-threading.
> However, for threading to make sense, the slaves on same physical
> machine have to give up their CPU usage, and this doesn't seem to be the
> case (top shows them running at close to 100%). Is there another MPI
> routine that polls for data and then gives up its time-slice?
>
> Any other suggestions?

I ran into a similar problem some time ago. My situation seems similar
to yours:
  1. the data in the MPI application has a to-and-fro nature.
  2. I cannot afford an MPI process that consumes 100% cpu
     while doing nothing.

My solution was to link two extra routines with my (FORTRAN)
application. These routines intercept mpi_recv and mpi_send, test the
status of the request, and sleep if it is not ready. The sleep time
has an exponential curve; it has a start value, factor, and maximum
value.

I made no source code changes to my application. When I include these
two routines at link time, the load from the application changes from
2.0 to 1.0

I use these with OpenMPI-1.2.8.

I have not tried -mca yield_when_idle 1; which may not be in 1.2.8.
Not sure.

Hope that helps
Douglas.

-- 
  Douglas Guptill                       voice: 902-461-9749
  Research Assistant, LSC 4640          email: douglas.guptill_at_[hidden]
  Oceanography Department               fax:   902-494-3877
  Dalhousie University
  Halifax, NS, B3H 4J1, Canada