Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] using OpenMPI + SGE in a heterogeneous network
From: John Hearns (john.hearns_at_[hidden])
Date: 2008-06-06 19:07:05


On Fri, 2008-06-06 at 17:56 +0100, SLIM H.A. wrote:
> Hi
>
> I want to use SGE to run jobs on a cluster with mx and infiniband nodes.
> By dividing the nodes into two host groups SGE will submit to either
> interconnect.
>
> The interconnect can be specified in the mpirun command with the --mca
> btl parameter. However users would then have to decide at runtime which
> of the two interconnects should be used, but if the job has to queue
> they just want the first available interconnect.

If you are just using hostgroups, how do you stop SGE from assigning
jobs across two hostgroups then? IE. if the user does not make an
explicit request for @infinihosts or @myrihosts SGE will allocate a set
of free hosts - and that could be a mix of the two types.

As Reuti says you need those wildcard PEs.

Or, as you say, just let the user decide when submitting the job.

ps. there is another way of doing the selection, other than hostgroups.
Set up two complexes, infiniband and myrinet (say) which are Boolean.
Then for each exec host set complex_values infiniband=true
complex_values myrinet=true
The user submits with -l infiniband
I'm not sure if the syntax -l infiniband | myrinet
meaning infiniband or myrinet is valid. Reuti?