Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] backends and plugins
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-08-07 05:02:59


Antoine Rougier finished his internship recently so here's a summary of
what he did in the "backends" branch. For the record, the goal of his
work was to explore how to change our backends into proper plugins so
that we can easily avoid hard dependencies between the hwloc core and
external libraries such as CUDA, libpci, ...

He defined three types of backends:
* "Base" is a normal OS backend such as Linux/Solaris/... This guy
manages PU, Cores, Sockets, ... Only one of these can be used at the
same time.
* "Additional" is what is added to "Base" backends. This is libpci, and
things added inside PCI devices (CUDA, ...). Multiple of these may be
used at the same time. They are likely invoked sequentially after the
Base backend discovery.
* "Global" is a special case for XML, Synthetic and Custom, which
replaces both Base and Additional. It's not clear that this type will be
needed in the end, "Base" with a special flag to disable "Additional"
backends might be enough.

During init(), all existing backends are placed on a list of "available"
backends. And a list of "enabled" backends is initialized to empty.
Between init() and load(), calling set_xml(), set_synthetic(),
set_fsroot(), ... will append some backends to the "enabled" list
(actually two lists, one for Base, one for Additional).
During load(), we check whether some Base backends are "enabled".
Otherwise we enable the default one of the current OS. Then we actually
discover things using the (unique) enabled Base backend followed by all
enabled Additional backends.

Aside from the main "discover" callback, backends may also define some
callbacks to be invoked when new object are created. The main example is
Linux creating "OS devices" when a new "PCI device" is added by the PCI
backend. CUDA could use that too to fill GPU PCI devices. This is not
strictly needed since adding these devices could still be done later,
once the PCI backend is done. We'll see.

I'll work on putting all this in a branch soon.

All this is about making interaction between backends nicer. Once this
is done, we'll be able to make actual plugins out of all this. One
problem that will come is that some backends are almost directly used
from outside the core. For instance exporting a topology to XML is a
public API call going down to XML plugin. lstopo and hwloc-ps using
Linux-specific tid_get_cpubind() calls causes similar problems.

Instead of allowing random API calls into plugin internals, we could
keep these backends internal, i.e. not making them plugins. At least for
OS backends, it makes sense. "synthetic" and "custom" also have no
reason to be pluginified either, they depend on nothing.

XML would like to be a plugin so that we stop depending on libxml2, but
we have an intermediate level to ease this. The main XML functions do
not depend on libxml2, they can remain internal and call either libxml2
or our minimalistic no-libxml2 support underneath. So we can keep the
common code and the minimalistic support internal, and only move the
libxml2-specific callbacks to a plugin.

Summary of what plugins we could have in the end:
* for main backends:
  + internal: synthetic, xml-core, custom, linux, solaris, ...
  + plugins: pci, cuda, display, ...
* for "low-level" xml backends:
  + internal: minimalistic xml support
  + plugin: libxml2
* and maybe lstopo backends one day
  + internal: console, txt, fig, windows
  + plugin: cairo

Brice