[libvirt] [PATCH 2/6] Added public API to enable post-copy migration
Cristian KLEIN
cristian.klein at cs.umu.se
Thu Sep 25 09:42:16 UTC 2014
On 2014-09-24 14:32, Jiri Denemark wrote:
> On Tue, Sep 23, 2014 at 16:09:57 +0200, Cristian Klein wrote:
>> Added a new `libvirt` migration flag `VIR_MIGRATE_POSTCOPY` and the
>> necessary code to pass this flag to qemu as migration capability.
>>
>> Signed-off-by: Cristian Klein <cristian.klein at cs.umu.se>
>> ---
>> include/libvirt/libvirt.h.in | 1 +
>> src/libvirt.c | 7 +++++++
>> src/qemu/qemu_migration.c | 43 +++++++++++++++++++++++++++++++++++++++++++
>> src/qemu/qemu_migration.h | 3 ++-
>> 4 files changed, 53 insertions(+), 1 deletion(-)
>>
>> diff --git a/include/libvirt/libvirt.h.in b/include/libvirt/libvirt.h.in
>> index 6371b7b..bdc33c6 100644
>> --- a/include/libvirt/libvirt.h.in
>> +++ b/include/libvirt/libvirt.h.in
>> @@ -1225,6 +1225,7 @@ typedef enum {
>> VIR_MIGRATE_ABORT_ON_ERROR = (1 << 12), /* abort migration on I/O errors happened during migration */
>> VIR_MIGRATE_AUTO_CONVERGE = (1 << 13), /* force convergence */
>> VIR_MIGRATE_RDMA_PIN_ALL = (1 << 14), /* RDMA memory pinning */
>> + VIR_MIGRATE_POSTCOPY = (1 << 15), /* enable (but don't start) post-copy */
>> } virDomainMigrateFlags;
>>
>>
>> diff --git a/src/libvirt.c b/src/libvirt.c
>> index 1a285ca..33aeafa 100644
>> --- a/src/libvirt.c
>> +++ b/src/libvirt.c
>> @@ -5265,6 +5265,7 @@ virDomainMigrateDirect(virDomainPtr domain,
>> * automatically when supported).
>> * VIR_MIGRATE_UNSAFE Force migration even if it is considered unsafe.
>> * VIR_MIGRATE_OFFLINE Migrate offline
>> + * VIR_MIGRATE_POSTCOPY Enable (but don't start) post-copy
>> *
>> * VIR_MIGRATE_TUNNELLED requires that VIR_MIGRATE_PEER2PEER be set.
>> * Applications using the VIR_MIGRATE_PEER2PEER flag will probably
>> @@ -5291,6 +5292,12 @@ virDomainMigrateDirect(virDomainPtr domain,
>> * can use either VIR_MIGRATE_NON_SHARED_DISK or
>> * VIR_MIGRATE_NON_SHARED_INC as they are mutually exclusive.
>> *
>> + * If you want to enable post-copy migration you must set the
>> + * VIR_MIGRATE_POSTCOPY flag. Once migration is active, you may
>> + * start post-copy by calling virDomainMigrateStartPostCopy.
>> + * When to start post-copy is entirely left to the user, libvirt
>> + * only implements the necessary mechanism.
>> + *
>> * In either case it is typically only necessary to specify a
>> * URI if the destination host has multiple interfaces and a
>> * specific interface is required to transmit migration data.
>> diff --git a/src/qemu/qemu_migration.c b/src/qemu/qemu_migration.c
>> index a5bd825..bd1f2d6 100644
>> --- a/src/qemu/qemu_migration.c
>> +++ b/src/qemu/qemu_migration.c
>> @@ -1799,6 +1799,45 @@ qemuMigrationSetOffline(virQEMUDriverPtr driver,
>>
>>
>> static int
>> +qemuMigrationSetPostCopy(virQEMUDriverPtr driver,
>> + virDomainObjPtr vm,
>> + qemuDomainAsyncJob job)
>
> s/ // in the two lines above to fix indentation.
Will fix in next version of the patch. Thanks for pointing out "make
syntax-check".
>> +{
>> + qemuDomainObjPrivatePtr priv = vm->privateData;
>> + int ret;
>> +
>> + if (qemuDomainObjEnterMonitorAsync(driver, vm, job) < 0)
>> + return -1;
>> +
>> + ret = qemuMonitorGetMigrationCapability(
>> + priv->mon,
>> + QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY);
>
> QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY is not defined anywhere in this
> patchset. It seems you forgot to update qemu_monitor.[ch].
Sorry about that, I divided the patches incorrectly. Will be fixed in
PATCH v2.
>> +
>> + if (ret < 0) {
>> + goto cleanup;
>> + } else if (ret == 0) {
>> + if (job == QEMU_ASYNC_JOB_MIGRATION_IN) {
>> + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s",
>> + _("Post-copy migration is not supported by "
>> + "target QEMU binary"));
>> + } else {
>> + virReportError(VIR_ERR_ARGUMENT_UNSUPPORTED, "%s",
>> + _("Post-copy migration is not supported by "
>> + "source QEMU binary"));
>> + }
>> + ret = -1;
>> + goto cleanup;
>> + }
>> +
>> + ret = qemuMonitorSetMigrationCapability(
>> + priv->mon,
>> + QEMU_MONITOR_MIGRATION_CAPS_POSTCOPY);
>> +
>> + cleanup:
>> + qemuDomainObjExitMonitor(driver, vm);
>> + return ret;
>> +}
>> +static int
>> qemuMigrationSetCompression(virQEMUDriverPtr driver,
>> virDomainObjPtr vm,
>> qemuDomainAsyncJob job)
>> @@ -3580,6 +3619,10 @@ qemuMigrationRun(virQEMUDriverPtr driver,
>> if (flags & VIR_MIGRATE_RDMA_PIN_ALL &&
>> qemuMigrationSetPinAll(driver, vm,
>> QEMU_ASYNC_JOB_MIGRATION_OUT) < 0)
>> +
>> + if (flags & VIR_MIGRATE_POSTCOPY &&
>> + qemuMigrationSetPostCopy(driver, vm,
>> + QEMU_ASYNC_JOB_MIGRATION_OUT) < 0)
>> goto cleanup;
>>
>> if (qemuDomainObjEnterMonitorAsync(driver, vm,
>
> I'd expect similar thing would need to be done in the Prepare phase on
> destination... However, if destination does not need to set the
> capability, we at least need to check if destination QEMU supports it
> and report failure from Prepare if it doesn't. And the
> QEMU_ASYNC_JOB_MIGRATION_IN branch in qemuMigrationSetPostCopy would be
> unreachable in that case.
It's up to the source qemu (through the migration protocol) to activate
post-copy on the destination qemu. So there are no other steps to be
done on the source.
I have mixed feelings about having libvirt check the necessary
capability on the destination. Personally, I would prefer qemu to return
an error like "post-copy unavailable" when the "migrate" command is
issued, as there might be more factors that influence the availability
of post-copy. For example, it's not sufficient for the destination qemu
to support post-copy, but the destination kernel also needs userfault
support.
Dave, I just tried migration from the post-copy qemu to master qemu and
it fails unexpectedly. The "migrate" command return no error, but a
"failed" migration status is returned later. Weirdly, this also happens
when I initiate non-post-copy migration. Do you happen to know what is
the qemu policy regarding migration? What would be the best way to
determine if a certain type of migration is feasible, while at the same
time returning a meaningful error to the user?
>> diff --git a/src/qemu/qemu_migration.h b/src/qemu/qemu_migration.h
>> index e7a90c3..349c9c4 100644
>> --- a/src/qemu/qemu_migration.h
>> +++ b/src/qemu/qemu_migration.h
>> @@ -41,7 +41,8 @@
>> VIR_MIGRATE_COMPRESSED | \
>> VIR_MIGRATE_ABORT_ON_ERROR | \
>> VIR_MIGRATE_AUTO_CONVERGE | \
>> - VIR_MIGRATE_RDMA_PIN_ALL)
>> + VIR_MIGRATE_RDMA_PIN_ALL | \
>> + VIR_MIGRATE_POSTCOPY)
>>
>> /* All supported migration parameters and their types. */
>> # define QEMU_MIGRATION_PARAMETERS \
--
Cristian Klein, PhD
Post-doc @ Umeå Universitet
http://www8.cs.umu.se/~cklein
More information about the libvir-list
mailing list