This probably shows my lack of understanding as to how OpenMPI
negotiates the connectivity between nodes when given a choice
of interfaces but anyway:
does dasher have any network interfaces that vixen does not?
The scenario I am imgaining would be that you ssh into dasher
from vixen using a "network" that both share and similarly, when
you mpirun from vixen, the network that OpenMPI uses is constrained
by the interfaces that can be seen from vixen, so you are fine.
However when you are on dasher, mpirun sees another interface which
it takes a liking to and so tries to use that, but that interface
is not available to vixen so the OpenMPI processes spawned there
terminate when they can't find that interface so as to talk back
to dasher's controlling process.
I know that you are no longer working with VMs but it's along those
lines that I was thinking: extra network interfaces that you assume
won't be used but which are and which could then be overcome by use
of an explicit
--mca btl_tcp_if_exclude virbr0
or some such construction (virbr0 used as an example here).
Kevin M. Buckley Room: CO327
School of Engineering and Phone: +64 4 463 5971
Victoria University of Wellington