I assume you mean something like mca_coll_foo_init_query() for your
initialization function. And I'm guessing you're exchanging some sort
of address information for your network here?
What I actually did in my collective component was use PML's modex
(module exchange) facility, defined in
ompi/mca/pml/base/pml_base_module_exchange.h. I think all of the BTLs
use this, so look there if you would like to see examples.
The fact that this is associated with the PML is a bit of a historical
artifact; it is perfectly reasonable to use the modex stuff from a coll
Another facility you might be interested in is ORTE's DSS (data
pack/unpack) and RML (remote messaging). The BTLs tend to use this for
out-of-band communication necessary for lazy connection establishment.
The headers you want are orte/dss/dss.h and orte/mca/rml/rml.h. Again,
many of the BTLs use this, so look there for examples.
Hope this helps!
Georg Wassen wrote:
> I'm working on a collective component and need point-to-point
> communication during module-initialization.
> As BTL is initialized prior to the collectives, I tried to use send and
> recv like MPI_Send/_Recv do:
> err = MCA_PML_CALL(send(buf, size, MPI_CHAR, to_id,
> COLL_SCI_TAG, MCA_PML_BASE_SEND_STANDARD, comm));
> This works for a total of 32 Bytes (8 times 4 Bytes) to each receiver.
> If I increase the number of processes, recv just doesn't return after 8
> recv-calls. (Further, this communication is extremely slow.)
> When I try to use MPI_Send/_Recv themselves, an error "MPI not yet
> initialized" occurs.
> Long story short: is there a way to communicate during
> mca_coll_*_module_init between different processes?
> (I don't want to use TCP/IP-sockets while Open MPI should be able to do
> this more portable.)
> Thanks for your help,
> Georg Wassen.
> devel mailing list