Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] How can I tell (open-)mpi about the HW topology of mysystem?
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2009-10-26 17:47:22

The short answer is that OMPI currently does not remap ranks during
MPI_CART_CREATE, even if you pass reorder==1. :-\

The reason is because we've had very little requests to do so.

However, we do have the good foresight (if I do say so myself ;-) ) to
make the MPI topology system be a plugin in Open MPI. The only plugin
for this system is currently the "do nothing" plugin, but it would
*not* be difficult to write one that actually did something meaningful
in your torus.

If you're interested, I'd be happy to explain how to do it (and we
should probably move to the devel list). OMPI doesn't require too
much framework code; I would guess that the majority of the code would
actually be implementing whatever algorithms you wanted for your
torus. Heck, you could even write a blind-and-dumb algorithm that
just looks up tables in files based on hostnames in your torus.

On Oct 23, 2009, at 7:54 AM, Luigi Scorzato wrote:

> Hi everybody,
> The short question is: How can I tell (open-)mpi about the HW
> topology of my system?
> The longer form is the following, I have a cluster which is
> physically connected in a 3D torus topology (say 5x3x2). The nodes
> have names: node_000, node_001, ... node_421. I can use a rankfile to
> assign a fix MPI rank to each node. E.g:
> rank 0 = node_000
> rank 1 = node_001
> rank 2 = node_010
> ...
> However, in general, nothing forces e.g. MPI_Cart_create() to build
> the 3D grid I want i.e. coord[node_ijk] =(i,j,k) rather than, say
> coord[node_000] =(0,0,0), coord[node_001] =(1,0,0), coord[node_010] =
> (2,0,0) ..., which would be wrongly mapped to the physical topology.
> How can I bind at least MPI_Cart_create() to the topology I want? Of
> course it would be nice to use an MPI-compliant procedure, if it
> exists. If not, I am also happy with something that works at least
> with some version of open-mpi.
> Note: For some reason too long to explain I cannot rely on a system
> that tests the the connections at the beginning. But the is no reason
> to do these tests, since I know my topology exactly.
> Thanks in advance for any help!
> Luigi
> _______________________________________________
> users mailing list
> users_at_[hidden]

Jeff Squyres