Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] dynamic rules
From: Roman Martonak (r.martonak_at_[hidden])
Date: 2010-01-15 07:54:33


On my machine I need to use dynamic rules to enforce the bruck or pairwise
algorithm for alltoall, since unfortunately the default basic linear
algorithm performs quite poorly on my
Infiniband network. Few months ago I noticed that in case of VASP,
however, the use of dynamic
rules via --mca coll_tuned_use_dynamic_rules 1 -mca
coll_tuned_dynamic_rules_filename dyn_rules
has no effect at all. Later it was identified that there was a bug
causing the dynamic rules to
apply only to the MPI_COMM_WORLD but not to other communicators. As
far as I understand, the bug
was fixed in openmpi-1.3.4. I tried now the openmpi-1.4 version and
expected that tuning of alltoall via dynamic
rules would work, but there is still no effect at all. Even worse, now
it is not even possible to use static rules
(which worked previously) such as -mca coll_tuned_alltoall_algorithm
3, because the code would crash (as discussed in the list recently).
When running with --mca coll_base_verbose 1000, I get messages like

[compute-0-0.local:08011] coll:sm:comm_query (0/MPI_COMM_WORLD):
intercomm, comm is too small, or not all peers local; disqualifying
myself
[compute-0-0.local:08011] coll:base:comm_select: component not available: sm
[compute-0-0.local:08011] coll:base:comm_select: component available:
sync, priority: 50
[compute-0-3.local:26116] coll:base:comm_select: component available:
self, priority: 75
[compute-0-3.local:26116] coll:sm:comm_query (1/MPI_COMM_SELF):
intercomm, comm is too small, or not all peers local; disqualifying
myself
[compute-0-3.local:26116] coll:base:comm_select: component not available: sm
[compute-0-3.local:26116] coll:base:comm_select: component available:
sync, priority: 50
[compute-0-3.local:26116] coll:base:comm_select: component not available: tuned
[compute-0-0.local:08011] coll:base:comm_select: component available:
tuned, priority: 30

Is there now a way to use other alltoall algorithms instead of the
basic linear algorithm in openmpi-1.4.x ?

Thanks in advance for any suggestion.

Best regards

Roman Martonak