Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] hwloc with Xen system support - some initial code
From: Andrew Cooper (andrew.cooper3_at_[hidden])
Date: 2014-01-01 16:47:40

On 01/01/2014 21:15, Brice Goglin wrote:
> Le 01/01/2014 21:55, Andrew Cooper a écrit :
>> But Xen itself has none of this information directly. In all cases,
>> hwloc should defer to the native PCI method, which is why I
>> explciticly didn't exclude it.
> Ok good then!
> By the way, I didn't test the code, I only compiled tested on a machine
> without Xen but with patched Xen libs, and loading hwloc hit the error
> path in instantiate()
> Brice

I tested the code insofar as running it repeatedly on my test box, and
finding a few of that larger/more interesting boxes in our test pool.
As said at the root of this thread, the diagrams drawn match my
expectation of how the servers are supposedly laid out.

At the moment, my list of activities is:
* See about getting plugins working (./configure is still wanting to
build Xen support in, rather than making it a plugin)
* Get NUMA memory and distances recorded (as the information is already
* Get offline cpus working.

I am in two minds about that patch to libxc.

On the one hand, libxenctrl is a fairly lightweight library (no
dependencies), whose purpose is to be a thin wrapper to basic Xen
controls. However, it explicitly has an unstable API which is free to
change in the future, and I can make no guarentee that the API of those
functions wont change (especially as I have a plan to remove the
ambiguity between online/offline cpus and their topology location).

The other option which I discounted initially was to use libxenlight.
libxl explicitly has a stable API (but unstable ABI). It comes with a
host of dependencies (libxlutil, libxenstore, libxenguest, libyajl,
libuuid, libbz2, liblzma, libz off the top of my head, and probably more).

Furthermore, libxc exported the information in the format I wanted to
consume it in, rather than libxl which translates it to a less useful
format as part of retrieving the information.

As the Xen part of hwloc is going to be a plugin, using libxl might not
be so bad, and having a stable API would certainly be good (even if it
means doing a reverse transform on the information received from libxl).

For the moment, I shall continue using libxc as it easier and already