[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] PATCH: Disable QEMU drive caching



Mark McLoughlin wrote:
i.e. with write-back caching enabled, the IDE  protocol makes no
guarantees about when data is committed to disk.

So, from a protocol correctness POV, qemu is behaving correctly with
cache=on and write-back caching enabled on the disk.

Yes, the host page cache is basically a big on-disk cache.

For SCSI, an unordered queue is advertised. Again, everything depends on whether or not write-back caching is enabled or not. Again, perfectly happy to take patches here.

Queue ordering and write-back caching sound like very different things.
Are they two distinct SCSI options, or ...?

Yes.

Surely an ordered queue doesn't do much help prevent fs corruption if
the host crashes, right? You would still need write-back caching
disabled?

You need both. In theory, a guest would use queue ordering to guarantee that certain writes made it to disk before other writes. Enabling write-through guarantees that the data is actually on disk. Since we advertise an unordered queue, we're okay from a safety point-of-view but for performance reasons, we'll want to do ordered queuing.

More importantly, the most common journaled filesystem, ext3, does not enable write barriers by default (even for journal updates). This is how it ship in Red Hat distros.

i.e. implementing barriers for virtio won't help most ext3 deployments?

Yes, ext3 doesn't use barriers by default. See http://kerneltrap.org/mailarchive/linux-kernel/2008/5/19/1865314

And again, if barriers are just about ordering, don't you need to
disable caching anyway?

Well, virtio doesn't have a notion of write-caching. My thinking is that we ought to implement barriers via fdatasync because posix-aio already has an op for it. This would effectively use barriers as a point to force something on disk. I think this would take care of most of the data corruption issues since the cases where the guest cares about data corruption would be handled (barriers should be used for journal writes and any O_DIRECT write for instance, although, yeah, not the case today with ext3).

So there is no greater risk of corrupting a journal in QEMU than there
is on bare metal.

This is the bit I really don't buy - we're equating qemu caching to IDE
write-back caching and saying the risk of corruption is the same in both
cases.

Yes.

But doesn't qemu cache data for far, far longer than a typical IDE disk
with write-back caching would do? Doesn't that mean you're far, far more
likely to see fs corruption with qemu caching?

It caches more data, I don't know how much longer it cases than a typical IDE disk. The guest can crash and that won't cause data loss. The only thing that will really cause data loss is the host crashing so it's slightly better than write-back caching from that regard.

Or put it another way, if we fix it by implementing the disabling of
write-back caching ... users running a virtual machine will need to run
"hdparam -W 0 /dev/sda" where they would never have run it on baremetal?

I don't see it as something needing to be fixed because I don't see that the exposure is significantly greater for a VM than for a real machine.

And let's take a step back too. If people are really concerned about this point, let's introduce a sync=on option that opens the image with O_SYNC. This will effectively make the cache write-through without the baggage associated with O_DIRECT.

While I object to libvirt always setting cache=off, I think sync=on for IDE and SCSI may be reasonable (you don't want it for virtio-blk once we implement proper barriers with fdatasync I think).

Regards,

Anthony Liguori

Cheers,
Mark.



[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]