Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [hwloc-devel] backends and plugins
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-08-07 07:06:20


On Aug 7, 2012, at 5:02 AM, Brice Goglin wrote:

> Antoine Rougier finished his internship recently so here's a summary of
> what he did in the "backends" branch. For the record, the goal of his
> work was to explore how to change our backends into proper plugins so
> that we can easily avoid hard dependencies between the hwloc core and
> external libraries such as CUDA, libpci, ...

In general, this sounds most excellent. Some specific comments, below.

> Aside from the main "discover" callback, backends may also define some
> callbacks to be invoked when new object are created. The main example is
> Linux creating "OS devices" when a new "PCI device" is added by the PCI
> backend. CUDA could use that too to fill GPU PCI devices. This is not
> strictly needed since adding these devices could still be done later,
> once the PCI backend is done. We'll see.

This is a nifty idea. Is the idea that callback can be registered to be fired when a specific PCI vendor / device ID are found?

> All this is about making interaction between backends nicer. Once this
> is done, we'll be able to make actual plugins out of all this. One
> problem that will come is that some backends are almost directly used
> from outside the core. For instance exporting a topology to XML is a
> public API call going down to XML plugin. lstopo and hwloc-ps using
> Linux-specific tid_get_cpubind() calls causes similar problems.

Another possibility is deprecating the old functions and making new functions, like hwloc_export_topo_to_file(..., HWLOC_FILE_FORMAT_XML), or something like that. I.e., call a dispatch routine before the actual back-end routine -- this would preserve the abstraction/plugin barrier. (you mention another viable possibility below, too -- I mention this one only for completeness)

I don't know if that's attractive or not, but I offer the possibility. :-)

> Instead of allowing random API calls into plugin internals, we could
> keep these backends internal, i.e. not making them plugins. At least for
> OS backends, it makes sense. "synthetic" and "custom" also have no
> reason to be pluginified either, they depend on nothing.

It might be nice to view all plugins as the same -- regardless of whether they are internal (i.e., part of libhwloc) or external (i.e., a standalone DSO). That way, the majority of the core code doesn't have to know/care whether plugins are internal or external.

It would also allow slurping external plugins to be internal, which will be fairly important for embedded mode. A specific case which has come up for this multiple times is when higher-level MPI bindings packages (e.g., Python) dlopen libmpi into a private namespace. When OMPI then tries to dlopen its own DSO/external plugins, they can't find the symbols in libmpi that they depend on (because libmpi is in a private namespace). Hence, OMPI has to be built in a slurp-all-plugins-to-be-internal mode to support such configurations.

As such, we'll need hwloc to also support this slurp-all-plugins-to-be-internal kind of mode, too.

I can help with the build mojo for this, if desired.

> XML would like to be a plugin so that we stop depending on libxml2, but
> we have an intermediate level to ease this. The main XML functions do
> not depend on libxml2, they can remain internal and call either libxml2
> or our minimalistic no-libxml2 support underneath. So we can keep the
> common code and the minimalistic support internal, and only move the
> libxml2-specific callbacks to a plugin.
>
> Summary of what plugins we could have in the end:
> * for main backends:
> + internal: synthetic, xml-core, custom, linux, solaris, ...
> + plugins: pci, cuda, display, ...
> * for "low-level" xml backends:
> + internal: minimalistic xml support
> + plugin: libxml2
> * and maybe lstopo backends one day
> + internal: console, txt, fig, windows
> + plugin: cairo
>
> Brice
>
> _______________________________________________
> hwloc-devel mailing list
> hwloc-devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/