Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] [EXTERNAL] Re: [OMPI svn-full] svn:open-mpi r29703 - in trunk: contrib/platform/iu/odin ompi/mca/btl/openib ompi/mca/btl/openib/connect
From: Shamis, Pavel (shamisp_at_[hidden])
Date: 2013-11-14 18:07:11

> So far as I can tell, the issue is one of blocking. The OOB handshake is now async - i.e., you post a non-blocking recv at the beginning of time, and then do a non-blocking send to the other side when you want to create a connection. The question is: how do you know when that connection is ready?

As you describe, the new behavior is identical to original one. We post non-blocking (persistent) receive during initialization. Later OMPI has barrier in the flow to ensure that all processes reached the point.
On first send, we use a non-blocking oob-send to initialize the connection (QPs). The receive triggers callback that handles the connection setup. OOB / XOOB communication semantics is a fully non-blocking.

We don't really block anywhere.
We use ompi_rte_recv_buffer_nb and ompi_rte_send_buffer_nb functions only.