And in practice the difference between FUNNELED and SERIALIZED will be
very small. The differences might emerge from thread-local state and
thread-specific network registration, but I don't see this being
required. Hence, for most purposes SINGLE=FUNNELED=SERIALIZED is
equivalent to NOMUTEX and MULTIPLE is MUTEX, where MUTEX refers to the
internal mutex required to make MPI reentrant.
On Wed, Oct 23, 2013 at 1:18 PM, Tim Prince <n8tm_at_[hidden]> wrote:
> On 10/23/2013 01:02 PM, Barrett, Brian W wrote:
> On 10/22/13 10:23 AM, "Jai Dayal" <dayalsoap_at_[hidden]> wrote:
> I, for the life of me, can't understand the difference between these two
> init_thread modes.
> MPI_THREAD_SINGLE states that "only one thread will execute", but
> MPI_THREAD_FUNNELED states "The process may be multi-threaded, but only the
> main thread will make MPI calls (all MPI calls are funneled to the main
> If I use MPI_THREAD_SINGLE, and just create a bunch of pthreads that dumbly
> loop in the background, the MPI library will have no way of detecting this,
> nor should this have any affects on the machine.
> This is exactly the same as MPI_THREAD_FUNNELED. What exactly does it mean
> with "only one thread will execute?" The openmpi library has absolutely zero
> way of knowng I've spawned other pthreads, and since these pthreads aren't
> actually doing MPI communication, I fail to see how this would interfere.
> Technically, if you call MPI_INIT_THREAD with MPI_THREAD_SINGLE, you have
> made a promise that you will not create any other threads in your
> application. There was a time where OSes shipped threaded and non-threaded
> malloc, for example, so knowing that might be important for that last bit of
> performance. There are also some obscure corner cases of the memory model
> of some architectures where you might get unexpected results if you made an
> MPI Receive call in an thread and accessed that buffer later from another
> thread, which may require memory barriers inside the implementation, so
> there could be some differences between SINGLE and FUNNELED due to those
> In Open MPI, we'll handle those corner cases whether you init for SINGLE or
> FUNNELED, so there's really no practical difference for Open MPI, but you're
> then slightly less portable.
> I'm asking because I'm using an open_mpi build ontop of infiniband, and the
> maximum thread mode is MPI_THREAD_SINGLE.
> That doesn't seem right; which version of Open MPI are you using?
> As Brian said, you aren't likely to be running on a system like Windows 98
> where non-thread-safe libraries were preferred. My colleagues at NASA
> insist that any properly built MPI will support MPI_THREAD_FUNNELED by
> default, even when the documentation says explicit setting in
> MPI_Init_thread() is mandatory. The statement which I see in OpenMPI doc
> says all MPI calls must be made by the thread which calls MPI_Init_thread.
> Apparently it will work if plain MPI_Init is used instead. This theory
> appears to hold up for all the MPI implementations of interest. The
> additional threads referred to are "inside the MPI rank," although I suppose
> additional application threads not involved with MPI are possible.
> users mailing list