Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] process placement with toruqe and OpenMP
From: Terry Frankcombe (terry_at_[hidden])
Date: 2008-01-29 14:19:21


> Ok so I ask the mpirun masters how would you do the following:
>
> I submit a job with torque (we use --with-tm) like the following:
>
> nodes=4:ppn=2
>
> My desired outcome is to have 1 mpi process per 2 cpus and use
> threaded blas (or my own OpenMP take your pick)

For the above, -np 4 --bynode should work.

<http://www.open-mpi.org/faq/?category=running#mpirun-scheduling>

>
> Our cluster has some 4 core machines thus the above job sometimes
> ends up looking like
>
> nodes=1:ppn=4+nodes=2:ppn=2
>
> The mpirun -bynode command will work in the case i get 4 nodes with
> only 2 cpus free. But if any machine other than the first machine is
> my node with 4 cores free given to me by moab, I would end up
> starting a extra process on the first node, where mpirun thinks
> another cpu is free, but that cpu is really to be used by OpenMP, and
> that the last process should be placed on the node that has 4 cpus free.
>
> I hope that wasn't to confusing, Its how to i launch hybrid jobs and
> make sure the process started by mpirun go where i want when my nodes
> have different core counts, and I am running via torque so using -H
> wont%