Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] multiple simultaneous topology inits?
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2011-03-01 05:00:02


Le 01/03/2011 10:46, Bernd Kallies a écrit :
> To calculate topology-based pinning schemes and do process pinning (like
> done e.g. by OpenMPI or MVAPICH2) this is too long, when every process
> (MPI task) or thread loads the topology in parallel. But exporting an
> XML topology and using this for this purpose is inaccepteable, when
> Linux cpusets are used, because one needs the topology of a subset of
> the machine depending on the caller context. What we currently do is to
> let only one process per machine load the topology, and distribute the
> essentials needed for pinning to the remaining processes.
>

You can still export one XML file per cpuset and load the right one in
each process. If the batch scheduler creates the cpuset, my feeling is
that the batch scheduler should create the XML file at the same time,
and do something like export HWLOC_XMLFILE=mycpuset.xml when launching
processes.

Also, in 1.2, we'll have a hwloc_topology_restrict() function which will
let you load the whole machine topology and then restrict it to whatever
part of it (a part is defined by a hwloc_cpuset_t). We'll need to make
sure that you'll have everything you need to get your cpuset's
hwloc_cpuset_t.

Brice