Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] merging plugins?
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-09-25 03:26:58


Le 25/09/2012 08:43, Samuel Thibault a écrit :
> Brice Goglin, le Tue 25 Sep 2012 07:41:48 +0200, a écrit :
>> Le 25/09/2012 01:42, Samuel Thibault a écrit :
>>>> 2) Plugin support
>>> One thing that doesn't seem implemented yet is to choose another OS core
>>> plugin, e.g. to use x86 detection on Linux instead of /proc or /sys
>>> detection. This will be the same kind of thing with likwid / servet
>>> -based OS core plugins. I have gotten the x86 detection code to get
>>> enabled with the attached code, which should be reproducable with other
>>> OSes which support CPU binding. How does it look like?
>>>
>> Indeed, I forget about this idea since Antoine didn't look at it at all.
>> Most of your code looks good. I just noted 2 points:
>> * Why not keeping the linux-x86 component always statically build? We
>> could even keep it inside topology-linux.c if they share a lot.
> How will it get detected by the component core? If we add it to the
> static component list, won't it be loaded by default?

Let me try to answer all your emails at once :)

On most OS, we already have multiple "core os" components, one native
(linux, ...) with priority 10 and the "noos" one with priority 0. If
nothing forces a specific component in the list (no env variable, no
set_xml, ...), we instantiate the first one (higher priority), which is
the native component when it exists.

So I assume that you're x86 support would have priority somewhere
between 0 and 10, and the env variable would let us force use it first.

>> * Your HWLOC_PLUGINS variable is not about loading plugins, it's about
>> enabling core components. If it's only about changing the core OS
>> component, a single value is enough (a single core OS component is
>> used). If it's also about forcing additional core components (pci, cuda,
>> for instance to change their order?), we'll likely have to check that
>> we're not instantiating the the same one multiple times.
> I guess I misunderstood when exactly which plugins are loaded, some
> documentation might be useful. Does hwloc just load all plugins it
> finds in its path? What makes it load the pci plugins, and not the
> linuxx86 plugin?
>

Everything is "loaded" and added to some list. Only one is enabled in
the core os list. All are enabled in the additional list (except if some
flag in the core os component disables things, that's what XML/synthetic
do to disable PCI).

pci (additional list) is always loaded, and it gets enabled except if
XML/synthetic set a flag.

linuxx86 (os list) only gets enabled if first in the list (priority) or
an env var forces it.

> Also, I guess that we probably want that some plugins be enabled by
> default if they are found (because they add useful information but
> depend on an external dep), and that some others are not enabled by
> default (because they add information, but e.g. to the expense of
> computation time, due to e.g. measurements etc.).

We have no way to force disable one specific additional component.
Could be a "expensive" flag or "cost" in the component struct with a env
var.
And a syntax such as pci,^servet to select manually.

> It could also be to use another PCI detection plugin that libpci.

Ok, two good reason to let us change the list the additional components.
Do we want to specifiy the "core OS" component and the "core additional"
list in the same env var?

There's currently no way to prevent two PCI backends to be used
simultaneously. Might use another flag so that only the first one
(libpci) or a specific one (env var) is enabled.

Brice