I wonder whether the OMPI 1.3 on OS X Leopard will have MPI threads enabled (the 1.2.8 version on Darwin doesn't)...
Is doing blocking communication in a separate thread better then
(At least as a workaround until the proper implementation gets improved)
At the moment, yes. OMPI's asynchronous progress is "loosely tested" (at best).
OMPI's threading support is somewhat stable for some devices (e.g., not OpenFabrics-based networks), but it's still somewhat new, so feedback would be welcome here.
I quickly modified my asynchronous code (Isend - Recv) to have a pthread doing Irecv - Rsend for point to point collective communication (which normally works fine for me). But still - on my wee laptop - the thread / blocking comm. combination is quite slow. I will play around with it to see whether it can be improved. Of course, laptop is not a good device for a very conclusive testing, but if things are too slow here, I wouldn't expect them to work great elsewhere.