Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Using (or not using) Torque/Moab under PBS Pro as the OpenMPI launcher
From: Brock Palen (brockp_at_[hidden])
Date: 2010-12-17 17:34:17


You can build openMPI without tm, which will disable it, or you can test first with a nasty option like:

           mpirun \
            --mca plm ^tm \
            --mca ras ^tm \
            --hostfile $PBS_NODEFILE \
            ./testmom

Brock Palen
www.umich.edu/~brockp
Center for Advanced Computing
brockp_at_[hidden]
(734)936-1985

On Dec 17, 2010, at 5:29 PM, David Whitaker wrote:

> Hi,
> At Cray, we have a a Cluster Compatibility Mode (CCM)
> which runs under PBSPro. If we do a qsub,
> we get an allocation of compute nodes and the first
> node in the list of allocated nodes is considered the
> "head node" of the "Cluster".
>
> The list of allocated nodes is contained in
> the file $PBS_NODEFILE.
>
> To launch a command on the head node we type:
> ccmrun COMMAND
>
> In the normal PBSPro environment, we launch
> the OpenMPI mpirun as:
> ccmrun ${OpenMPI_Install}/bin/mpirun <options ..>
> -hostfile host.list -np $NCPUS ./hello_mpi
>
> This all works great with the "mpirun" command
> launching on the "head node" and the the mpirun
> command using ssh/rsh to launch other processes
> on the other nodes in the virtual cluster. Everything
> works.
>
> On Cray sites, that use PBSPro/Torque/Moab, OpenMPI
> detects that Torque/Moab is present and attempts
> to use Torque/Moab to launch on other nodes. Because of
> the way Cray has Torque/moab configured, this doesn't work.
> This is not a problem with OpenMPI.
>
> Is there an option to OpenMPI to tell it to ignore
> Torque/Moab and use rsh/ssh and the -hostfile/hosts
> even if Torque/Moab is detected?? I've looked through
> the README and the FAQ but I haven't seen anything.
>
> Many thanks,
> Dave
>
>
>
> --
> CCCCCCCCCCCCCCCCCCCCCCFFFFFFFFFFFFFFFFFFFFFFFFFDDDDDDDDDDDDDDDDDDDDD
> David Whitaker, Ph.D. whitaker_at_[hidden]
> Aerospace CFD Specialist phone: (651)605-9078
> ISV Applications/Cray Inc fax: (651)605-9001
> CCCCCCCCCCCCCCCCCCCCCCFFFFFFFFFFFFFFFFFFFFFFFFFDDDDDDDDDDDDDDDDDDDDD
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>