Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] backends and plugins
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-08-07 07:06:20


On Aug 7, 2012, at 5:02 AM, Brice Goglin wrote:

> Antoine Rougier finished his internship recently so here's a summary of
> what he did in the "backends" branch. For the record, the goal of his
> work was to explore how to change our backends into proper plugins so
> that we can easily avoid hard dependencies between the hwloc core and
> external libraries such as CUDA, libpci, ...

In general, this sounds most excellent. Some specific comments, below.

> Aside from the main "discover" callback, backends may also define some
> callbacks to be invoked when new object are created. The main example is
> Linux creating "OS devices" when a new "PCI device" is added by the PCI
> backend. CUDA could use that too to fill GPU PCI devices. This is not
> strictly needed since adding these devices could still be done later,
> once the PCI backend is done. We'll see.

This is a nifty idea. Is the idea that callback can be registered to be fired when a specific PCI vendor / device ID are found?

> All this is about making interaction between backends nicer. Once this
> is done, we'll be able to make actual plugins out of all this. One
> problem that will come is that some backends are almost directly used
> from outside the core. For instance exporting a topology to XML is a
> public API call going down to XML plugin. lstopo and hwloc-ps using
> Linux-specific tid_get_cpubind() calls causes similar problems.

Another possibility is deprecating the old functions and making new functions, like hwloc_export_topo_to_file(..., HWLOC_FILE_FORMAT_XML), or something like that. I.e., call a dispatch routine before the actual back-end routine -- this would preserve the abstraction/plugin barrier. (you mention another viable possibility below, too -- I mention this one only for completeness)

I don't know if that's attractive or not, but I offer the possibility. :-)

> Instead of allowing random API calls into plugin internals, we could
> keep these backends internal, i.e. not making them plugins. At least for
> OS backends, it makes sense. "synthetic" and "custom" also have no
> reason to be pluginified either, they depend on nothing.

It might be nice to view all plugins as the same -- regardless of whether they are internal (i.e., part of libhwloc) or external (i.e., a standalone DSO). That way, the majority of the core code doesn't have to know/care whether plugins are internal or external.

It would also allow slurping external plugins to be internal, which will be fairly important for embedded mode. A specific case which has come up for this multiple times is when higher-level MPI bindings packages (e.g., Python) dlopen libmpi into a private namespace. When OMPI then tries to dlopen its own DSO/external plugins, they can't find the symbols in libmpi that they depend on (because libmpi is in a private namespace). Hence, OMPI has to be built in a slurp-all-plugins-to-be-internal mode to support such configurations.

As such, we'll need hwloc to also support this slurp-all-plugins-to-be-internal kind of mode, too.

I can help with the build mojo for this, if desired.

> XML would like to be a plugin so that we stop depending on libxml2, but
> we have an intermediate level to ease this. The main XML functions do
> not depend on libxml2, they can remain internal and call either libxml2
> or our minimalistic no-libxml2 support underneath. So we can keep the
> common code and the minimalistic support internal, and only move the
> libxml2-specific callbacks to a plugin.
>
> Summary of what plugins we could have in the end:
> * for main backends:
> + internal: synthetic, xml-core, custom, linux, solaris, ...
> + plugins: pci, cuda, display, ...
> * for "low-level" xml backends:
> + internal: minimalistic xml support
> + plugin: libxml2
> * and maybe lstopo backends one day
> + internal: console, txt, fig, windows
> + plugin: cairo
>
> Brice
>
> _______________________________________________
> hwloc-devel mailing list
> hwloc-devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/