Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network
From: Reuti (reuti_at_[hidden])
Date: 2008-06-06 18:33:33


Am 07.06.2008 um 00:06 schrieb SLIM H.A.:

> The btl interface is used. The nodes in both subclusters have the
> same architecture so no need for two binaries. The MPI libraries
> for the interconnect can be selected at runtime, for example
> through the value of the OMPI_MCA_btl environment variable. However
> that variable should then somehow be set once SGE has decided on
> which subcluster to run the job, thus avoiding it to be set in
> advance when submitting. I tried a couple of things like different
> PEs but there does not seem to be an elegant solution; I mean
> something completely transparent to the user like

A transparent solution would be to create a "starter_method" in the
parallel queue, which will check the node that it's running on and
hence you know which type of interface to use. Export the variable
and exec the real script; i.e. something like:

#!/bin/sh
if [ "${HOSTNAME%%[0-9]*}" = "node_ib" ]; then
     export OMPI_MCA_btl=tcp
else
     export OMPI_MCA_btl=openib
fi
exec "$@"

I think you get the idea of the script. Maybe you have some special
names for these machines. Another indicator would be the $PE or
$QUEUE of SGE. If you want to be sure to get only nodes of one and
the same type (but you don't care which at submission time), you will
need to setup something like:

http://gridengine.info/articles/2006/02/14/grouping-jobs-to-nodes-via-
wildcard-pes

anyway.

HTH - Reuti

>
> mpirun -np $NSLOTS executable
>
> as I have seen in some of the OpenMPI FAQs.
>
> Thanks
>
> Henk
>
> From: users-bounces_at_[hidden] on behalf of Reuti
> Sent: Fri 6/6/2008 6:47 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] using OpenMPI + SGE in a heterogeneous
> network
>
> Am 06.06.2008 um 19:31 schrieb Patrick Geoffray:
>
> > SLIM H.A. wrote:
> >> I would be grateful for any advice
> >
> > Just to check, you are not using the MTL for MX, right ? Only the
> BTL
> > interface allow to choose between several devices at run time.
>
> At least there would be the option to built two binaries and decide
> at runtime which to use in your jobscript - depending on the machine
> you landed on.
>
> -- Reuti
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users