On 10/23/2013 01:02 PM, Barrett, Brian W wrote:
> On 10/22/13 10:23 AM, "Jai Dayal" <dayalsoap_at_[hidden]
> <mailto:dayalsoap_at_[hidden]>> wrote:
> I, for the life of me, can't understand the difference between
> these two init_thread modes.
> MPI_THREAD_SINGLE states that "only one thread will execute", but
> MPI_THREAD_FUNNELED states "The process may be multi-threaded, but
> only the main thread will make MPI calls (all MPI calls are
> funneled to the main thread)."
> If I use MPI_THREAD_SINGLE, and just create a bunch of pthreads
> that dumbly loop in the background, the MPI library will have no
> way of detecting this, nor should this have any affects on the
> This is exactly the same as MPI_THREAD_FUNNELED. What exactly does
> it mean with "only one thread will execute?" The openmpi library
> has absolutely zero way of knowng I've spawned other pthreads, and
> since these pthreads aren't actually doing MPI communication, I
> fail to see how this would interfere.
> Technically, if you call MPI_INIT_THREAD with MPI_THREAD_SINGLE, you
> have made a promise that you will not create any other threads in your
> application. There was a time where OSes shipped threaded and
> non-threaded malloc, for example, so knowing that might be important
> for that last bit of performance. There are also some obscure corner
> cases of the memory model of some architectures where you might get
> unexpected results if you made an MPI Receive call in an thread and
> accessed that buffer later from another thread, which may require
> memory barriers inside the implementation, so there could be some
> differences between SINGLE and FUNNELED due to those barriers.
> In Open MPI, we'll handle those corner cases whether you init for
> SINGLE or FUNNELED, so there's really no practical difference for Open
> MPI, but you're then slightly less portable.
> I'm asking because I'm using an open_mpi build ontop of
> infiniband, and the maximum thread mode is MPI_THREAD_SINGLE.
> That doesn't seem right; which version of Open MPI are you using?
As Brian said, you aren't likely to be running on a system like Windows
98 where non-thread-safe libraries were preferred. My colleagues at
NASA insist that any properly built MPI will support MPI_THREAD_FUNNELED
by default, even when the documentation says explicit setting in
MPI_Init_thread() is mandatory. The statement which I see in OpenMPI
doc says all MPI calls must be made by the thread which calls
MPI_Init_thread. Apparently it will work if plain MPI_Init is used
instead. This theory appears to hold up for all the MPI implementations
of interest. The additional threads referred to are "inside the MPI
rank," although I suppose additional application threads not involved
with MPI are possible.