[libvirt] [PATCH v6 0/2] Add support for qcow2 cache

[John Ferlan]

On 09/27/2017 08:42 AM, Pavel Hrdina wrote:
> On Wed, Sep 27, 2017 at 02:24:12PM +0200, Pavel Hrdina wrote:
>> On Tue, Sep 26, 2017 at 05:06:37PM -0400, John Ferlan wrote:
>>>
>>>
>>> On 09/18/2017 11:45 PM, Liu Qing wrote:
>>>> Qcow2 small IO random write performance will drop dramatically if the l2
>>>> cache table could not cover the whole disk. This will be a lot of l2
>>>> cache table RW operations if cache miss happens frequently.
>>>>  
>>>> This patch exports the qcow2 driver parameter
>>>> l2-cache-size/refcount-cache-size, first added in Qemu 2.2, and
>>>> cache-clean-interval, first added in Qemu 2.5, in libvirt.
>>>>
>>>> change since v4: removed unnecessary cache error check
>>>>
>>>> Liu Qing (2):
>>>>   conf, docs: Add qcow2 cache configuration support
>>>
>>> According to what you added for the v5 - there's an upstream bz that
>>> could be associated with this work.  Is this something that should be
>>> added to the commit message so that we can "track" where the idea comes
>>> from?
>>>
>>> Beyond that - I agree setting policy is not the business we want to be
>>> in. I just figured I'd throw it out there. I didn't read all of the qemu
>>> links from the bz about thoughts on this - too many links too much to
>>> follow not enough time/desire to think about it.
>>>
>>> I'm not against pushing these patches, but would like to at least make
>>> sure Peter isn't 100% against them. I agree the attributes are really
>>> painful to understand, but it's not the first time we've added some
>>> attribute that comes with a beware of using this unless you know what
>>> you're doing.
>>
>> I'm still not convinced that this is something we would like to expose
>> to users.  It's way too low-level for libvirt XML.  I like the idea to
>> allow 'max' as possible value in QEMU and that could be reused by
>> libvirt.
>>
>> One may argue that having two options as "max storage" or
>> "max performace" isn't good enough and they would like to be able to
>> tune it for they exact needs.  Most of the users don't care about the
>> specifics and they just need a performance or not.
>>
>> I attempted to introduce "polling" feature [1] for iothreads but that
>> was a bad idea and too low-level for libvirt users and configuring exact
>> values for qcow2 cache seems to be the same case.
>>
>> Adding an element/attribute into XML where you would specify whether you
>> care about performance or not isn't a policy.  Libvirt will simply use
>> some defaults or configure the maximum performance.  For fine-grained
>> tuning you can use <qemu:arg value='...'/> like it was suggested for
>> "polling" feature.
>>
>> Pavel
> 
> Ehm, I forgot to include the link to the "polling" discussion:
> 
> [1] <https://www.redhat.com/archives/libvir-list/2017-February/msg01084.html>
> 

I scanned the .1 discussion... it seems as though there were adjustments
made to the default iothreads to make them useful or at least enabled by
default a better set of values which could be disabled via some
<qemu:arg value='-newarg'/> setting.

>From the discussion thus far for qcow2 cache the difference here is that
the default values QEMU uses have been inadequate for real world type
usages and it doesn't appear convergence is possible because the
possible adjustments has vary degrees of issues. I think it's nicely
summed up here :

https://bugzilla.redhat.com/show_bug.cgi?id=1377735#c2 and
https://bugzilla.redhat.com/show_bug.cgi?id=1377735#c3

Following the link in the bz from comment 5 to the qemu patches shows
only a couple of attempts at trying something before the idea was
seemingly dropped. I looked forward and only recently was the idea
resurrected, but perhaps in perhaps a bit different format.

https://lists.gnu.org/archive/html/qemu-block/2017-09/msg00635.html

In any case, I've sent an email to the author of those patches to get
insights regarding adding exposure from libvirt is a good or not and to
determine if the new proposal somehow overrides any settins that would
be made. It'd be awful if a value was provided, but how that value is
used gets changed...

John