[libvirt] [RFC] Interface for disk hotadd/remove

[Wolfgang Mauerer]

Hi,

On Thu, Aug 13, 2009 at 8:31 PM, Daniel P. Berrange<berrange at redhat.com> wrote:
> On Thu, Aug 13, 2009 at 12:54:00PM +0200, Wolfgang Mauerer wrote:
>> I'm currently interested in implementing hard disk hot-add and -remove support
>> for qemu (as opposed to controller-based hotplugging), and this brings up the
>> question how to best support this feature in libvirt. Many SCSI-Controllers in
>> real machines, for instance, allow to add and remove disks (without adding or
>> removing the controller itself) while the system is up and running, so it
>> would be nice to emulate this in a virtual machine. I'm focusing on
>> qemu on the backend side, but the problem is not related to this
>> particular backend. Rather, the question is how to integrate such
>> a feature best into libvirt.
>>
>> Before implementing the functionality, it would be great to hear the
>> community's opinion which route to take wrt. the interface. Essentially, I can
>> see two options:
>>
>> - Naturally, there are virDomain{At,De}tachDevice, but the pair implements
>>  drive-hotadding via adding a new controller with an attached hard disk to
>>  the system. By extending the XML description of the drive with a parameter
>>  that specifies whether controller- or disk-based hotplugging is to be
>>  performed, it would be possible to implement the desired functionality,
>>  whilst preserving compatibility with existing semantics. Removing the drive
>>  would then require another new parameter in the XML description to identify
>>  the drive on the controller, which does not really prettify the thing.
>>
>> - Extend the API with a new method (for instance virDomainDiskAttach) that
>>  takes a hard disk description, a controller identifier, and a parameter that
>>  identifies the disk on the controller.
>
> I don't think its desirable to extend the API. The virDomainAttachDevice
> API's XML parameter is intended to take any XML element that is valid
> inside the domain <devices> XML section.

okay, that's good to know.

> So the key to deciding how to deal with hotplug, is to first decide how
> to represent disk controllers in the domain XML.
>
> At boot time, if you list multiple  SCSI disks in the XML, you get a single
> controller with multiple disks attached. As such the current semantics of
> the hotplug implementation for SCSI are divergant from the semantics at
> boot time. This has the consequence that if you boot a guest with 2 SCSI
> drives, then hot-attach one more you get 2 controllers, 3 disk. If you
> then shutdown & boot the guest again, you'll have 1 controller, 1 disks.
>
> This says to me that the hotplug implementation for QEMU SCSI should be
> fixed so that if you supply <disk> it adds a disk to the existing SCSI
> controller. Obviously it requires some QEMU support for this first.

Qemu support for this is already there (I was not quite clear on that,
the thing I'm currently adding additionally is hot-remove). I completely
agree that the current semantics of libvirt-controlled disk-hotplug in qemu
provide some possibilities for improvement, but I was not sure if it's
okay to change the semantics of a libvirt command whilst preserving its syntax.

> Thus my feeling would be todo something like adding a new <controller>
> element to represent a disk controller. Support hotplugging <controller>
> instances directly with obvious semantics.
>
> Then extend the <disk> schema to allow a controller name/identifier to
> be provided. If no controller is listed then plug the disk into the
> first available controller slot, otherwise use the explicitly requested
> controller. Never implicitily add new controllers, unless dealing with
> a legacy QEMU without the disk hotplug support that you're writing.

Sounds reasonable to me. Roughly, that would lead to something along
the lines of (for a SCSI host on the PCI bus)

<controller type="scsi" id="my_controller" >
    <bus addr="00:04"/>
</controller>

<disk type="scsi">
    <source file="disk.img"/>
    <controller id="my_controller" unit="0"/>
</disk>

This allows for specifying both the address of the controller on the
bus, and to identify the disk on the controller -- this is important for
hot-remove if multiple "physically" identical disks are attached to a single
controller.

If virDomainAttachDevice is fed with a <disk> element containing
an explicitely specified controller id, libvirt can then simply add the
disk if the controller exists, or add controller+disk if the controller
is not yet present on the machine. If the controller is not specified
and none is present in the system, follow the current behaviour. If
no controller is specified, but a controller is present, just add the
disk, not controller+disk, breaking current behaviour. For hot-remove,
do the reverse action (though I'll consider under which circumstances
it is desirable to remove the controller when a disk is supposed to be
removed, but these details are best discussed once I've done the
code, I suppose).  Does what I've outlined correspond to your idea?

Thanks,

Wolfgang