Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Method for worker to determine its "rank" on a single machine?
From: Gus Correa (gus_at_[hidden])
Date: 2010-12-10 13:26:41


Hi David

For what it is worth, the method suggested by
Terry Dontje and Richard Troutmann is what is used in several
generations of climate coupled models that we've been using for the
past 8+ years.

The goals are slightly different from yours:
they cut across logical boundaries
(i.e. who's atmosphere, who's ocean, etc),
whereas you want to cut across physical boundaries
(i.e. belonging to the same computer,
as diffuse as the notion of "same computer" can be these days).

The variants of this technique that I know of
are slightly different from Terry's suggestion:
they don't split the (MPI_COMM_WORLD) communicator,
but create additional sub-communicators instead.
However, the idea is the same.

The upside of this technique, as Terry and Richard point out,
is portability.
These models have been run in IBM Blue Genes using the IBM MPI,
on Kraken and Jaguar (Cray XT5 or XT6?) using whatever MPI they
have there, and I can even run them in our modest Beowulf clusters,
using OpenMPI or MVAPICH2, or even MPICH2.
All MPI calls are completely standard, hence the code is portable.
If the code had calls to the "orte" layer
(or to "P4" in the old days of MPICH) for instance, it wouldn't be.

If portability, specially portability across MPI variants, is important
to you, you may think of implementing the functionality you need
this way.

And to the MPI insiders/developers, a plea from a mere user:
Whatever you take to the Forum,
please keep this functionality (creating new communicators, splitting
old ones, getting processor name, etc) in the standard,
although the extensions suggested by Ralph Castain and Eugene Loh
would be certainly welcome.

Cheers,
Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------

David Mathog wrote:
>> The answer is yes - sort of...
>>
>> In OpenMPI, every process has information about not only its own local
> rank, but the local rank of all its peers regardless of what node they
> are on. We use that info internally for a variety of things.
>> Now the "sort of". That info isn't exposed via an MPI API at this
> time. If that doesn't matter, then I can tell you how to get it - it's
> pretty trivial to do.
>
> Please tell me how to do this using the internal information.
>
> For now I will use that to write these functions (which might at some
> point correspond to standard functions, or not)
>
> my_MPI_Local_size(MPI_Comm comm, int *lmax, int *lactual)
> my_MPI_Local_rank(MPI_Comm comm, int *lrank)
>
> These will return N for lmax, a value M in 1->N for lactual, and a value
> in 1->M for lrank, for any worker on a machine corresponding to a
> hostfile line like:
>
> node123.cluster slots=N
>
> As usual, this could get complicated. There are probably race
> conditions on lactual vs. lrank as the workers start, but I'm guessing
> the lrank to lmax relationship won't have that problem. Similarly, the
> meaning of "local" is pretty abstract. For now all that is intended is
> "a group of equivalent cores within a single enclosure, where
> communication between them is strictly internal to the enclosure, and
> where all have equivalent access to the local disks and the network
> interface(s)". Other ways to define "local" might make more sense on
> more complex hardware.
>
> Another function that logically belongs with these is:
>
> my_MPI_Local_list(MPI_Comm comm, int *llist, int *lactual)
>
> I don't need it now, but can imagine applications that would. This
> would return the (current) lactual value and the corresponding list of
> rank numbers of all the local workers. The array llist must be of size
> lmax.
>
>
> Thanks,
>
> David Mathog
> mathog_at_[hidden]
> Manager, Sequence Analysis Facility, Biology Division, Caltech
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users