Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Mapping ranks to hosts (from MPI error messages)
From: Dave Love (d.love_at_[hidden])
Date: 2014-03-27 18:59:18


Reuti <reuti_at_[hidden]> writes:

> Do all of them have an internal bookkeeping of granted cores to slots
> - i.e. not only the number of scheduled slots per job per node, but
> also which core was granted to which job? Does Open MPI read this
> information would be the next question then.

OMPI works with the bindings it's handed via orted (if the processes are
started that way).

>> My understanding is that Torque delegates to OpenMPI the process placement and binding (beyond the list of nodes/cpus available for
>> the job).

Can't/doesn't torque start the MPI processes itself? Otherwise, yes,
since orted gets the binding.

>> My guess is that OpenPBS behaves the same as Torque.
>>
>> SLURM and SGE/OGE *probably* have pretty much the same behavior.
>
> SGE/OGE: no, any binding request is only a soft request.

I don't understand that. Does it mean the system-specific "strict" and
"non-strict" binding in hwloc, in which case I don't see how UGE can do
anything different?

> UGE: here you can request a hard binding. But I have no clue whether this information is read by Open MPI too.
>
> If in doubt: use only complete nodes for each job (which is often done
> for massively parallel jobs anyway).

There's no need with a recent SGE. All our jobs get core bindings --
unless they use all the cores, since binding them all is equivalent to
binding none -- and OMPI inherits them. See
<http://arc.liv.ac.uk/SGE/howto/sge-configs.html#_core_binding> for the
SGE+OMPI configuration.

-- 
Community Grid Engine:  http://arc.liv.ac.uk/SGE/