The difference is that MPI_Barrier or even MPI_Ibarrier is a crude tool for the job. It is likely to leave resources idle that could be doing productive work.

I agree it is part of the tool kit but if this kind of problem is significant enough so a textbook should cover it then I would advocate that people be taught where to look for the problem and to consider which tool to use.

The first step is to look for way to improve load balance.  The second step is to look for ways to impede as few processes as possible and only when they must be impeded.

I am also suggesting that there is a middle class of application that improves with semantically redundant barrier. Applications with a significant design flaw that it would be better to find and fix than to mask.

The MPI standard provides for a flow control to prevent eager message flooding but it does not provide a flow control for descriptor flooding. Descriptor flooding may even bring down a job and the standard does not oblige the MPI implementation to prevent that.  This problem was understood from the beginning but it cannot be solved within the semantic rules for non-blocking Send/Recv. The options were to give the problem to the application writer or to say "A non-blocking operation will return promptly unless there is a resource shortage".

              Dick


Dick Treumann  -  MPI Team          
IBM Systems & Technology Group
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846         Fax (845) 433-8363


users-bounces@open-mpi.org wrote on 09/10/2010 10:27:02 AM:

> [image removed]
>
> Re: [OMPI users] MPI_Reduce performance
>
> Eugene Loh
>
> to:
>
> Open MPI Users
>
> 09/10/2010 10:30 AM
>
> Sent by:
>
> users-bounces@open-mpi.org
>
> Please respond to Open MPI Users
>
> Richard Treumann wrote:
>
> Hi Ashley
>
> I understand the problem with descriptor flooding can be serious in
> an application with unidirectional data dependancy. Perhaps we have
> a different perception of how common that is.
> Ashley speculated it was a "significant minority."  I don't know
> what that means, but it seems like it is a minority (most
> computations have causal relationships among the processes holding
> unbounded imbalances in check) and yet we end up seeing these exceptions.
> I think that adding some flow control to the application is a better
> solution than semantically redundant barrier.
> It seems to me there is no difference.  Flow control, at this level,
> is just semantically redundant synchronization.  A barrier is just a
> special case of that._______________________________________________
> users mailing list
> users@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/users