A call to MPI_Init allows the MPI library to return any level of thread support it chooses. This MPI 1.1 call does not let the application say what it wants and does not let the implementation reply with what it can guarantee.

If you are using only one MPI implementation and your code will never be run on another implementation you can check the user doc for that MPI implementation and see if says that you can do what you want to do. If your application is single threaded you can use MPI_Init and be portable.

If your application has threads, you should use MPI_Init_thread and check the response to be portable or to make sure you get thread safety from an MPI implementation that has both a thread safe and a thread unsafe mode. A thread unsafe mode can skip some locking and maybe give better performance.

The MPI standard wanted to allow for the possibility that some implementation of MPI would not be able to tolerate threads in the application at all so SINGLE was one answer that could be returned. For what you want to do, you need a library that can return FUNNELED or better. If you ask for FUNNELED and the MPI implementation you are using returns SINGLE it is telling you that using OpenMP threads is not allowed. It does not say why. It may work fine but the implementation is telling you not to do it and you cannot know if there are good reasons.

Consider a made up example:

Imagine some system supports Mutex lock/unlock but with terrible performance. As a work around, it offers a non-standard substitute for malloc called st_malloc (single thread malloc) that does not do locking. Normal malloc is also available and it does use locking. The documentation for this system says that it is safe to use both st_malloc and malloc in a single threaded application because the only difference is that st_malloc skips the locking. It also warns that if one thread calls st_malloc while another calls malloc, heap corruption is likely.

Next imagine that the MPI implementation uses st_malloc to provide best performance and the OpenMP threads use malloc. When this particular MPI implementation returns SINGLE, it really means SINGLE. If there is only one application thread using regular malloc it is safe but if there is a malloc call on one thread while the main thread is in an MPI call, heap corruption is likely.




Dick Treumann - MPI Team
IBM Systems & Technology Group
Dept X2ZA / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363


Inactive hide details for Yuanyuan ZHANG ---03/03/2010 07:34:07 PM---Hi guys, Thanks for your help, but unfortunately I am stilYuanyuan ZHANG ---03/03/2010 07:34:07 PM---Hi guys, Thanks for your help, but unfortunately I am still not clear.


From:

Yuanyuan ZHANG <zhang.yuanyuan@jp.fujitsu.com>

To:

Open MPI Users <users@open-mpi.org>

Date:

03/03/2010 07:34 PM

Subject:

Re: [OMPI users] MPI_Init() and MPI_Init_thread()

Sent by:

users-bounces@open-mpi.org





Hi guys,

Thanks for your help, but unfortunately I am still not clear.

> You are right Dave, FUNNELED allows the application to have multiple
> threads but only the man thread calls MPI.
My understanding is that even if I use SINGLE or MPI_Init, I can still
have multiple threads if I use OpenMP PARALLEL directive, and only
the main thread makes MPI calls. Am I correct?

> An OpenMP/MPI hybrid program that makes MPI calls only in between parallel
> sections is usually a FUNNELED user of MPI
For an OpenMP/MPI hybrid program, if I only want to make MPI calls using
the main thread, ie., only in between parallel sections, can I just use
SINGLE or MPI_Init? What's the benefit of FUNNELED?

Thanks.


_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users