Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Optimal distribution of low and high CPU processes on a heterogeneous cluster
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-05-05 09:28:25

On Apr 25, 2008, at 2:27 PM, Roopesh Ojha wrote:

> For our application, some processes (manager and the datastreams)
> won't
> take much CPU power while others will. Our application first assigns
> the
> non-compute bound processes and then the high CPU power ones.
> Clearly we
> would like to be able to assign the former processes to our two slower
> machines. How can we get MPI to achieve this?

Sorry for the delay in replying.

I'm not entirely clear on exactly what you want to do. I think you're
saying that you have 4 nodes:


And you want to schedule some slow procs and fast procs. I think you
should be able to do something like:

mpirun -np 2 --host slownode1,slownode2 slowproc : \
        -np 2 --host fastnode1,fastnode2 fastproc

> What is the best way to
> use
> slot and max-slot commands in the hostfile to optimize performance?

These options are not really about performance; they're used for
distributing processes across nodes. In your case, those are tightly
integrated, but note that they're really different things.

*Never* tell OMPI that there are more slots than are actually
available on a host.

> How
> does max-slot behave when some node is listed more than once in the
> hostfile?

IIRC, it's additive.

> Is there any way to control where processes are deployed apart from
> the
> basic bynode or byslot round robin schedule?

In the v1.2 series, no. In the v1.3 series, some new patterns were
just implemented within the last week or three -- docs have not been
written up yet, but Ralph Castain described some of them on the devel
list. Have a look through those archives on the web for more detail
(I'd get the details wrong if I tried to explain them).

Jeff Squyres
Cisco Systems