Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Collective operations and synchronization
From: George Bosilca (bosilca_at_[hidden])
Date: 2009-03-23 17:41:20


Unfortunately even the MPI_Barrier doesn't guarantee a synchronous
exit on all processes. There is no such thing in the MPI and there is
no way to implement such a synchronization primitive in general (if
one take in account metrics such as performance or scalability).

In this particular context the MPI_Allreduce offers you exactly the
same kind of synchronization as the MPI_Barrier. Moreover, all non
rooted collectives (with the exception of the v versions) imply a
synchronous behavior simply because all processes involved in the
collective have to participate with some data.

   george.

On Mar 23, 2009, at 17:11 , Ralph Castain wrote:

> Just one point to emphasize - Eugene said it, but many times people
> don't fully grasp the implication.
>
> On an MPI_Allreduce, the algorithm requires that all processes -
> enter- the call before anyone can exit.
>
> It does -not- require that they all exit at the same time.
>
> So if you want to synchronize on the -exit-, as your question
> indicated, then you must add the MPI_Barrier as you describe.
>
> Ralph
>
>
> On Mar 23, 2009, at 3:01 PM, Eugene Loh wrote:
>
>> Shaun Jackman wrote:
>>
>>> I've just read in the Open MPI documentation [1]
>>
>> That's the MPI spec, actually.
>>
>>> that collective operations, such as MPI_Allreduce, may
>>> synchronize, but do not necessarily synchronize. My algorithm
>>> requires a collective operation and synchronization; is there a
>>> better (more efficient?) method than simply calling MPI_Allreduce
>>> followed by MPI_Barrier?
>>
>> MPI_Allreduce is a case that actually "requires" synchronization in
>> that no participating process may exit before all processes have
>> entered. So, there should be no need to add additional
>> synchronization. A special case might be an MPI_Allreduce of a 0-
>> length message, in which case I suppose an MPI implementation could
>> simple "do nothing", and the synchronization side-effect would be
>> lost.
>>
>> The MPI spec is mainly talking about a "typical" collective where
>> one could imagine a process exiting before some processes have
>> entered. E.g., in a broadcast or scatter, the root could exit
>> before any other process has entered the operation. In a reduce or
>> gather, the root could enter after all other processes have
>> exited. For all-to-all, allreduce, or allgather, however, no
>> process can exit before all processes have entered, which is the
>> synchronization condition effected by a barrier. (Again, null
>> message lengths can change things.)
>>
>>> [1] http://www.mpi-forum.org/docs/mpi21-report-bw/node85.htm
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users