Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] hwloc_cpuset_t to cgroups cpuset conversion
From: Rayson Ho (raysonlogin_at_[hidden])
Date: 2012-06-19 02:32:52


hwloc already has code (linux-libnuma.h) for hwloc_cpuset_t to libnuma
data type conversion. Does it make sense to add functions to convert
hwloc_cpuset_t types to the cpuset string format used by Linux's
cgroups?

The cpuset format used by cgroups is not difficult to handle (in fact,
the user interface is same as/very similar to the older cpuset
implementation). Eg. create a cpuset that includes CPU 2 & 3 for
OGS/Grid Engine:

# echo 2-3 > /sys/fs/cgroup/cpuset/gridengine/cpuset.cpus

Ref: http://www.kernel.org/doc/Documentation/cgroups/cpusets.txt

We are pinging the list because we use hwloc for topology binding, and
we also added cgroups support (for a lot of things) in OGS/Grid
Engine:

http://blogs.scalablelogic.com/2012/05/grid-engine-cgroups-integration.html

We are planning to integrate hwloc & cgroups (and skip libnuma)... And
we want to choose which method we use at runtime. While hwloc can be
used by non-root applications, the applications can modify the
affinity mask themselves and thus they can escape from the "cpuset"
(as least with the current hwloc implementation that uses
sched_setaffinity). The cgroups cpuset support is only available to
root, but once binded, batch jobs running as non-root could not escape
from the cpuset.

Rayson

==================================================
Open Grid Scheduler - The Official Open Source Grid Engine
http://gridscheduler.sourceforge.net/