[libvirt] [RFC PATCH 0/2] nodeinfo: PPC64: Fix topology and siblings info on capabilities and nodeinfo

[Andrea Bolognani]

On Wed, 2016-07-20 at 13:49 +0100, Daniel P. Berrange wrote:
> > IIRC doing something like this was proposed at some point in
> > the past, but was rejected on the ground that existing tools
> > assume that 1) CPUs listed in the NUMA topology are online
> > and 2) you can pin vCPUs to them. So they would try to pin
> > vCPUs to eg. CPU 1 and that will fail.
> 
> We could easily deal with this by adding a flag to the
> virConnectGetCapabilities() API  eg VIR_CONNECT_CAPS_ALL_CPUS
> so we only show the expanded full CPU list if an app opts
> in to it.

Okay.

> > Additionally, this doesn't tell us anything about whether any
> > host CPU can run a guest CPU: given the above configuration,
> > on ppc64, we know that CPU 1 can run guest threads even though
> > it's offline because CPU 0 is online, but the same isn't true
> > on x86.
> > 
> > So we would end up needing three new boolean properties:
> > 
> >   - online - whether the CPU is online
> >   - can_run_vcpus - whether the CPU can run vCPUs
> >   - can_pin_vcpus - whether vCPUs can be pinned to the CPU
> 
> These two can*vcpus props aren't telling the whole story
> because they are assuming use of KVM - TCG or LXC guests
> would not follow the same rules for runability here.
> 
> This is why I think the host capabilities XML should focus
> on describing what hardware actually exists and its state,
> and not say anything about guest runnability.
> 
> Historically we've let apps figure out the runnability
> of pCPUs vs guest vCPUs, but even on x86 it isn't as simple
> as you/we make it out to be.
> 
> For example, nothing here reflects the fact that the host
> OS could have /sys/fs/cgroups/cpuset/cpuset.cpus configured
> to a subset of CPUs.  So already today on x86, just because
> a CPU is listed in the capabilities XML, does not imply that
> you can run a guest on it.
> 
> So I think there's a gap for exposing information about
> runability of guests vs host CPUs, that does not really
> fit in the host capabilities. Possibly it could be in the
> guest capabilities, but more likely it would need to be
> a new API entirely.

Why wouldn't domcapabilities be a good place? We already
have some related information (the <vcpu> tag).

In any case, regardless of whether or not it will ultimately
be part of domcapabilities, I guess a good starting point is
to sketch out how the xml would look like. I'm thinking of
something like

  <cpu id='0' runnable='yes' pinnable='yes' run_group='0-3'/>
  <cpu id='1' runnable='yes' pinnable='no' run_group='0-3'/>
  <cpu id='2' runnable='yes' pinnable='no' run_group='0-3'/>
  <cpu id='3' runnable='yes' pinnable='no' run_group='0-3'/>

where 'runnable' tells you whether the CPU can run vCPUs,
'pinnable' whether vCPUs can be pinned, and 'run_group' tells
you what CPUs the pinned vCPUs will actually run on? On x86
it would look simpler:

  <cpu id='0' runnable='yes' pinnable='yes' run_group='0'/>
  <cpu id='1' runnable='yes' pinnable='yes' run_group='1'/>

I think we don't need to add information that can already be
obtained from existing capabilities, such as the siblings
list.

The value of 'runnable' etc. would be calculated taking into
account the architecture, the hypervisor, and host
configuration details such as the ones you mentioned above.

Does the above look reasonable? Anything I'm missing?

-- 
Andrea Bolognani / Red Hat / Virtualization