Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] hwloc with Xen system support - some initial code
From: Andrew Cooper (andrew.cooper3_at_[hidden])
Date: 2014-01-06 09:31:39


On 06/01/14 14:10, Brice Goglin wrote:
> Hello,
>
> Here's a new patchset that fixes the remaining problems on my side.
> 01 is basically the same I sent earlier.
> 02 and 03 export two core functions to plugins
> 04 is basically the same I sent earlier, without the hacky #include
> <private/private.h> that was waiting for 02-03
> 05 lets me check that topology-xen.c builds when libxen isn't available
> (useful for regression testing all backends when changing internals,
> used with "make check -C tests/ports")
>
> The Xen component seems to work fine as a plugin. The Xen lib dependency
> is only in the plugin:
> $ ldd utils/.libs/lstopo | grep xen
> $ ldd src/.libs/libhwloc.so | grep xen
> $ ldd src/.libs/hwloc_xen.so | grep xen
> libxenctrl-4.3.so => /usr/lib/libxenctrl-4.3.so (0x00007f3de693c000)
>
> You can try with
> make && make install && HWLOC_XEN=1 utils/lstopo
> Add HWLOC_PLUGINS_VERBOSE for details about plugins being loaded. And
> HWLOC_COMPONENTS_VERBOSE for details components being enabled or not.
>
>
> A couple comments below.

Fantastic thanks.

I have a slightly improved version which does Numa node memory
annotations as well (in so far as Xen reports them, which currently
doesn't reflect reality - another bug raised against Xen)

I shall merge these patches and my current WIP to make a "xen hwloc v2"
branch for review.

>
>
> Le 01/01/2014 22:47, Andrew Cooper a écrit :
>> At the moment, my list of activities is:
>> * See about getting plugins working (./configure is still wanting to
>> build Xen support in, rather than making it a plugin)
>> * Get NUMA memory and distances recorded (as the information is already
>> available)
>> * Get offline cpus working.
>>
>> I am in two minds about that patch to libxc.
>>
>> On the one hand, libxenctrl is a fairly lightweight library (no
>> dependencies), whose purpose is to be a thin wrapper to basic Xen
>> controls. However, it explicitly has an unstable API which is free to
>> change in the future, and I can make no guarentee that the API of those
>> functions wont change (especially as I have a plan to remove the
>> ambiguity between online/offline cpus and their topology location).
> Is there a #define that we check to easily detect which API is available
> without having multiple API checks in configure?

No, which is one of the pain-points for using it.

>
>> The other option which I discounted initially was to use libxenlight.
>> libxl explicitly has a stable API (but unstable ABI). It comes with a
>> host of dependencies (libxlutil, libxenstore, libxenguest, libyajl,
>> libuuid, libbz2, liblzma, libz off the top of my head, and probably more).
> According to you, in reality, how often will libxl be installed on
> existing platforms, compared to libxc?

The available hypercalls in Xen are only present in versions of Xen
which also have libxl, so the answer should be 100% of the time.
Whether libxl gets packaged properly is a different matter (I know that
debian is particularly crazy at the moment).

>
>> Furthermore, libxc exported the information in the format I wanted to
>> consume it in, rather than libxl which translates it to a less useful
>> format as part of retrieving the information.
>>
>> As the Xen part of hwloc is going to be a plugin, using libxl might not
>> be so bad, and having a stable API would certainly be good (even if it
>> means doing a reverse transform on the information received from libxl).
> We won't release hwloc v1.9 before a couple months. You can take some
> time to decide. I don't have a strong feeling about this choice.
>
> Brice

I will bring it up on xen-devel when post v2.

~Andrew