On Wed, Jul 21, 2010 at 02:10:53PM -0400, David Ronis wrote:
> I've got a mpi program on an 8-core box that runs in a master-slave
> mode. The slaves calculate something, pass data to the master, and
> then call MPI_Bcast waiting for the master to update and return some
> data via a MPI_Bcast originating on the master.
> One of the things the master does while the slaves are waiting is to
> make heavy use of fftw3 FFT routines which can support multi-threading.
> However, for threading to make sense, the slaves on same physical
> machine have to give up their CPU usage, and this doesn't seem to be the
> case (top shows them running at close to 100%). Is there another MPI
> routine that polls for data and then gives up its time-slice?
> Any other suggestions?
I ran into a similar problem some time ago. My situation seems similar
1. the data in the MPI application has a to-and-fro nature.
2. I cannot afford an MPI process that consumes 100% cpu
while doing nothing.
My solution was to link two extra routines with my (FORTRAN)
application. These routines intercept mpi_recv and mpi_send, test the
status of the request, and sleep if it is not ready. The sleep time
has an exponential curve; it has a start value, factor, and maximum
I made no source code changes to my application. When I include these
two routines at link time, the load from the application changes from
2.0 to 1.0
I use these with OpenMPI-1.2.8.
I have not tried -mca yield_when_idle 1; which may not be in 1.2.8.
Hope that helps
Douglas Guptill voice: 902-461-9749
Research Assistant, LSC 4640 email: douglas.guptill_at_[hidden]
Oceanography Department fax: 902-494-3877
Halifax, NS, B3H 4J1, Canada