The short question is: How can I tell (open-)mpi about the HW
topology of my system?
The longer form is the following, I have a cluster which is
physically connected in a 3D torus topology (say 5x3x2). The nodes
have names: node_000, node_001, ... node_421. I can use a rankfile to
assign a fix MPI rank to each node. E.g:
rank 0 = node_000
rank 1 = node_001
rank 2 = node_010
However, in general, nothing forces e.g. MPI_Cart_create() to build
the 3D grid I want i.e. coord[node_ijk] =(i,j,k) rather than, say
coord[node_000] =(0,0,0), coord[node_001] =(1,0,0), coord[node_010] =
(2,0,0) ..., which would be wrongly mapped to the physical topology.
How can I bind at least MPI_Cart_create() to the topology I want? Of
course it would be nice to use an MPI-compliant procedure, if it
exists. If not, I am also happy with something that works at least
with some version of open-mpi.
Note: For some reason too long to explain I cannot rely on a system
that tests the the connections at the beginning. But the is no reason
to do these tests, since I know my topology exactly.
Thanks in advance for any help!