[libvirt] The design choice for how to enable block I/O throttling function in libvirt

[Stefan Hajnoczi]

On Tue, Aug 30, 2011 at 2:46 PM, Adam Litke <agl at us.ibm.com> wrote:
> On Tue, Aug 30, 2011 at 09:53:33AM +0100, Stefan Hajnoczi wrote:
>> On Tue, Aug 30, 2011 at 3:55 AM, Zhi Yong Wu <zwu.kernel at gmail.com> wrote:
>> > I am trying to enable block I/O throttling function in libvirt. But
>> > currently i met some design questions, and don't make sure if we
>> > should extend blkiotune to support block I/O throttling or introduce
>> > one new libvirt command "blkiothrottle" to cover it or not. If you
>> > have some better idea, pls don't hesitate to drop your comments.
>>
>> A little bit of context: this discussion is about adding libvirt
>> support for QEMU disk I/O throttling.
>
> Thanks for the additional context Stefan.
>
>> Today libvirt supports the cgroups blkio-controller, which handles
>> proportional shares and throughput/iops limits on host block devices.
>> blkio-controller does not support network file systems (NFS) or other
>> QEMU remote block drivers (curl, Ceph/rbd, sheepdog) since they are
>> not host block devices.  QEMU I/O throttling works with all types of
>> -drive and therefore complements blkio-controller.
>
> The first question that pops into my mind is: Should a user need to understand
> when to use the cgroups blkio-controller vs. the QEMU I/O throttling method?  In
> my opinion, it would be nice if libvirt had a single interface for block I/O
> throttling and libvirt would decide which mechanism to use based on the type of
> device and the specific limits that need to be set.

Yes, I agree it would be simplest to pick the right mechanism,
depending on the type of throttling the user wants.  More below.

>> I/O throttling can be applied independently to each -drive attached to
>> a guest and supports throughput/iops limits.  For more information on
>> this QEMU feature and a comparison with blkio-controller, see Ryan
>> Harper's KVM Forum 2011 presentation:
>
>> http://www.linux-kvm.org/wiki/images/7/72/2011-forum-keep-a-limit-on-it-io-throttling-in-qemu.pdf
>
> From the presentation, it seems that both the cgroups method the the qemu method
> offer comparable control (assuming a block device) so it might possible to apply
> either method from the same API in a transparent manner.  Am I correct or are we
> suggesting that the Qemu throttling approach should always be used for Qemu
> domains?

QEMU I/O throttling does not provide a proportional share mechanism.
So you cannot assign weights to VMs and let them receive a fraction of
the available disk time.  That is only supported by cgroups
blkio-controller because it requires a global view which QEMU does not
have.

So I think the two are complementary:

If proportional share should be used on a host block device, use
cgroups blkio-controller.
Otherwise use QEMU I/O throttling.

Stefan