[Date Prev][Date Next] [Thread Prev][Thread Next]
[libvirt] vCPU hotplug roadmap (was: Minutes for KVM call 2013-01-15)
- From: Andreas Färber <afaerber suse de>
- To: qemu-devel qemu-devel <qemu-devel nongnu org>, KVM devel mailing list <kvm vger kernel org>
- Cc: Peter Maydell <peter maydell linaro org>, Eduardo Habkost <ehabkost redhat com>, Juan Quintela <quintela redhat com>, libvir-list redhat com, Benjamin Herrenschmidt <benh kernel crashing org>, Alexander Graf <agraf suse de>, Christian Borntraeger <borntraeger de ibm com>, qemu-ppc <qemu-ppc nongnu org>, Anthony Liguori <anthony codemonkey ws>, Igor Mammedov <imammedo redhat com>, David Gibson <david gibson dropbear id au>
- Subject: [libvirt] vCPU hotplug roadmap (was: Minutes for KVM call 2013-01-15)
- Date: Wed, 30 Jan 2013 11:58:56 +0100
Am 15.01.2013 17:16, schrieb Juan Quintela:
> * cpu hot plug
> - use qdev propierties conected to a set of socket objects (anthony)
> - cpusets are the wrong interface (anthony)
> - make a link between cpu <-> socket instead of a propierty?
> - how far are we from being able to describe a cpu with -device?
> (didn't heare the answer, andreas?)
> - perhaps the best approach?
> - After soft-freeze, exceptions depend on the maintainer
> - After hard-freeze, no exceptions
> -device don't require a bus, just an implementation detail, we can change that
> - use cpuset as an intermediate step until full vision is implemented
> - several approaches from where we are now, to have something before
> we get a full solution
> At this point, Andreas agreed to write a better summary of the
> discussion and suggestions O:-)
Got buried, here we go:
== vCPU hot-plug user interfaces ==
=== cpu_set ===
Previously available in qemu-kvm.git:
`cpu_set <n+1> online` via HMP
* Hides QOM/qdev implementation details (afaerber)
* Thus: Doesn't depend on QOM CPUState refactoring (imammedo)
* Opens a fast route to implementing vCPU unplug in KVM (imammedo)
* Unintrusive to add and easy to obsolete/remove in future (imammedo)
* Existing virt-test cases (afaerber)
* Supported by libvirt (imammedo)
* Prevents confusing guests by hot-plugging random mix of CPUs (agraf)
* Cannot express topologies (ehabkost)
=== device_add ===
`device_add driver=Haswell-x86_64-cpu id=qdevid`
[You can try this today and see it failing / not working.]
* QMP/HMP command available today and known to users (afaerber)
* Unified command for device and CPU hot-plug (imammedo)
* Would allow first doing thread-level vCPU hotplug (imammedo)
* Could be extended to support socket-level hot-plug (aliguori/imammedo)
* Operates on raw QOM type name unlike -cpu (afaerber)
* Needs support in libvirt for device_add driver=<CPU> (imammedo)
* libvirt needs means to enumerate CPU types (imammedo) => QMP? (AF)
* No CPU qbus (afaerber)
=> "should work" without (aliguori)
* CPU subclasses needed for identifying type name (afaerber/imammedo)
=> "Haswell-x86_64-cpu" does not exist yet, just "x86_64-cpu"
* CPU class_init for -cpu host requires KVM init (imammedo)
[suggestion by ehabkost to use kvm_arch_vcpu_init, WIP by afaerber]
* Conversion of CPU features to static properties needed (imammedo)
=> device_add driver=foo,level=x,xlevel=y,...
* Alternatively conversion to global properties (imammedo)
* Cements type names - rename for 1.4? (afaerber) => permissable (alig.)
[patches for arm, m68k, openrisc, unicore32 on list]
=== qom-set ===
`qom-set` via QMP w/ link<CPUSocket> property (aliguori)
Topology represented in QOM:
CPUSocket has-a CPUCore has-a CPUThread a.k.a. CPUState, or
CPUSocket links-to CPUCore links-to CPUThread a.k.a. CPUState
* No CPUSocket/CPUCore objects yet and may take a while to get there...
topology fields being moved to CPUState for 1.4 [done, more WIP]
* No decisions on canonical paths for CPUs: CPU? machine? unassigned?
* Duality of thread-level device types and socket-level? (afaerber)
=> fine to have, e.g., quad-core Xeon 500 device (aliguori)
* CPUState is no_user (afaerber)
=> need to generally drop no_user for QOM (aliguori)
=== libvirt ===
libvirt's XML topology modelling is closer to today's -smp than to the
desired QOM modelling:
`virsh setvcpus <domain> <n>`
== qom-cpu course of action (afaerber) ==
It was requested to have vCPU hot-plug in v1.5.
For device_add we need to move code from cpu_init() into QOM facilities.
=> QOM realize support would help [applied by aliguori]
=> cleanups piggy-backed onto CPU realizefn [applied to qom-cpu-next]
Agreement on goal of X86CPU subclasses, but conflicts how to get there:
* Refactor x86_def_t to X86CPUInfo for X86CPUClass class_init? (AF 2012)
* Refactor x86_def_t to X86CPU instance_init as done for arm?
* Refactor x86_def_t to class_inits? (afaerber)
-> heavy merge conflicts due to bug fixes / cleanups
Pro: We can get things into a consistent QOM'ish state across targets.
Con: We will refactor again on top for machine-compat properties.
* Keep x86_def_t within X86CPUClass as done for ppc? (WIP: afaerber)
=> smallest common denominator, separates x86 from cross-target work
APIC ID topology fixes are being reviewed for 1.4. [merged]
X86CPU wave 4 cleanups by Igor are being reviewed for 1.4. [merged]
Rename CPU types according to unified <name>-<arch>-cpu scheme for 1.4?
(aliguori: permissable) [patches on list]
VMState series by Juan being rebased - subset for 1.4, rest for 1.5.
[1.4 part on list, WIP for 1.5]
Remainder is considered 1.5 material, qom-cpu-next avail. during Freeze.
== Common issues (imammedo) ==
- back-port CPU hot-plug ACPI notification
- hot-plug is not allowed on SysBus:
- APIC that is created by CPU on hot-plug is a SysBus device,
so essentially APIC is hot-plugged to SysBus anyway which causes
abort in qdev_try_create() -> qdev_set_parent_bus(SysBus) ->
bus_add_child() [no hot-plug on SysBus], which is hard to fix
without allowing hot-plug on SysBus.
[AF: change the APIC to not be a SysBusDevice (like CPUState) then?]
- qdev_device_add() and qdev_try_create() assign SysBus by default to
device if no explicit bus was provided.
- qbus_find_recursive() in qdev_device_add() always returns SysBus
if last 2 args are NULL, which is case with busless CPU
- qdev_try_create() adds device SysBus if no bus was provided in
- vapic is not working with hot-plug in current master
(do not know why yet)
Experimental CPU hot-add tree using device_add implemented only for
That completes Igor's and my notes, amended by status updates of mine.
Sorry for the delay, I did try to address some of the discussed issues
in form of patches in the meantime though. ;)
CC'ing s390x, ppc and arm KVM folks that were not on the call for
feedback on their requirements and thoughts.
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg
[Date Prev][Date Next] [Thread Prev][Thread Next]