Reuti <reuti_at_[hidden]> writes:
> Do all of them have an internal bookkeeping of granted cores to slots
> - i.e. not only the number of scheduled slots per job per node, but
> also which core was granted to which job? Does Open MPI read this
> information would be the next question then.
OMPI works with the bindings it's handed via orted (if the processes are
started that way).
>> My understanding is that Torque delegates to OpenMPI the process placement and binding (beyond the list of nodes/cpus available for
>> the job).
Can't/doesn't torque start the MPI processes itself? Otherwise, yes,
since orted gets the binding.
>> My guess is that OpenPBS behaves the same as Torque.
>> SLURM and SGE/OGE *probably* have pretty much the same behavior.
> SGE/OGE: no, any binding request is only a soft request.
I don't understand that. Does it mean the system-specific "strict" and
"non-strict" binding in hwloc, in which case I don't see how UGE can do
> UGE: here you can request a hard binding. But I have no clue whether this information is read by Open MPI too.
> If in doubt: use only complete nodes for each job (which is often done
> for massively parallel jobs anyway).
There's no need with a recent SGE. All our jobs get core bindings --
unless they use all the cores, since binding them all is equivalent to
binding none -- and OMPI inherits them. See
<http://arc.liv.ac.uk/SGE/howto/sge-configs.html#_core_binding> for the
Community Grid Engine: http://arc.liv.ac.uk/SGE/