Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] Creating a topology generation method for Xen
From: Andrew Cooper (andrew.cooper3_at_[hidden])
Date: 2013-12-26 18:25:54

On 26/12/2013 22:52, Brice Goglin wrote:
> Hello,
> How would you like the user to switch from the fake/guest topology to
> the real/host topology in practice? Most applications may still want
> fake/guest topos (so that binding works etc) while admins and only some
> (advanced?) users will want the real/host topology.
> We can easily make Xen lower priority by default and use environment
> variables to raise/enable Xen manually. But I am not sure that's a
> convenient solution for the end user.
> If we add --enable-xen, you'd want to have two hwloc installations, a
> normal one and a xen-enabled one?
> Brice

I was hoping to put off that question.

There is an interesting development in progress at the moment to give a
VM some real topology information, based on where its vcpus are actually
pinned. At the moment by default, a vcpu is eligible to be scheduled on
any pcpu, and the memory is deliberately striped across all numa nodes
so the performance is equally wherever the vcpu is scheduled.

This was fine 10 years ago, but is a large performance penalty these
days, which is why there is a lot of work going on to rectify the
situation. As a result, I can foresee a proper usecase for the hwloc
binding using the VM-presented topology.

In hindsight, an --enable-xen option is probably silly, as I
(personally) would specifically not want two builds of hwloc in dom0.

Having said that, having a low priority and some environmental tweak is
somewhat inconvenient.

At the end of the day, the system level topology is almost certainly
going to be a read-only source of information. Therefore, it make most
sense for a host administrator to go out of their way when looking at
the system topology, rather than attempting to tune things inside the
dom0 fake topology.