[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [libvirt] QEMU interfaces for image streaming and post-copy block migration

On 09/07/2010 10:03 AM, Daniel P. Berrange wrote:
On Tue, Sep 07, 2010 at 08:41:44AM -0500, Anthony Liguori wrote:

We've got copy-on-read and image streaming working in QED and before
going much further, I wanted to bounce some interfaces off of the
libvirt folks to make sure our final interface makes sense.

Here's the basic idea:

A related topic is block migration.  Today we support pre-copy migration
which means we transfer the block device and then do a live migration.
Another approach is to do a live migration, and on the source, run a
block server using image streaming on the destination to move the device.

With QED, to implement this one would:

1) launch qemu-nbd on the source while the guest is running
2) create a qed file on the destination with copy-on-read enabled and a
backing file using nbd: to point to the source qemu-nbd
3) run qemu -incoming on the destination with the qed file
4) execute the migration
5) when migration completes, begin streaming on the destination to
complete the copy
6) when the streaming is complete, shut down the qemu-nbd instance on
the source
IMHO, adding further network sockets is the one thing we absolutely
don't want to do to migration. I don't much like the idea of launching
extra daemons either.

One of the use cases I'm trying to accommodate is migration to free resources. By launching a qemu-nbd daemon, we can kill the source qemu process and free up all of the associated memory.

This is a bit involved and we could potentially automate some of this in
qemu by launching qemu-nbd and providing commands to do some of this.
Again though, I think the question is what type of interfaces would
libvirt prefer?  Low level interfaces + recipes on how to do high level
things or higher level interfaces?
I think it should be done entirely within the main QEMU migration
socket. I know this isn't possible with the current impl, since it
is unidirectional, preventing the target sending the source requests
for specific data blocks. If we made migration socket bi-directional
I think we could do it all within qemu with no external helpers
or extra sockets

  1. Create empty qed file on the destination with copy on read
     enable backing file pointing to a special 'migrate:' protocol

Why not just point migration and nbd to a unix domain socket and then multiplex the two protocols at a higher level?

  2. Run qemu -incoming on the destination with with the qed file
  3. execute the migration
  4. when migration completes, target QEMU continues streaming blocks
     from the soruce qemu.
  5. when streaming is complete, source qemu can shutdown.

Both your original proposal and mine here seem to have a pretty
bad failure scenario though. After the cut-over point where the
VM cpus start running on the destination QEMU, AFAICT, any failure
on the source before block streaming complete leaves you dead in
the water.  The source VM no longer has up2date RAM contents and
the destination VM does not yet have a complete disk image.

Yes. It's a trade off. However, pre-copy doesn't really change your likelihood of catastrophic failure because if you were going to fail in the source, it was going to happen before you completed the block transfer anyway.

The advantage of post-copy is that you immediately free resources on the source so as a reaction to pressure from overcommit, it's tremendously useful.

I still think pre-copy has it's place though.


Anthony Liguori


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]