[Libguestfs] [PATCH] v2v: Output saved overlays in a machine-readable fashion

[Richard W.M. Jones]

On Tue, Oct 15, 2019 at 04:53:03PM +0000, Roman Kagan wrote:
> On Fri, Oct 11, 2019 at 01:29:53PM +0100, Richard W.M. Jones wrote:
> > Stepping back I think the problem is we're shoehorning features into a
> > tool (virt-v2v) which was designed to do cold conversions, and was
> > never meant to do either warm or in-place conversions.  The tool was
> > hacked a while back for in-place but that mode is very awkward to use
> > and we have never enabled it in RHEL for good reason.
>
> As I'm guilty for introducing --in-place, let me chime in with my 2c
> and humbly disagree with it being awkward to use.  I think the only
> thing that's awkward is its tight integration into virt-v2v; it'd
> make more sense being a separate tool whose sole purpose would be to
> tune the guest to be bootable/operational in the provided
> configuration.

My original comments there are rather rude.  What I should have said
is the use of the input metadata (eg. --inplace -i libvirtxml) as a
kind of configuration file for selecting the rcaps (eg. whether to
install virtio drivers) are somewhat non-obvious, although as you say
in the context of virt-v2v that's all that can be done.

I would - same as you - like to split out virt-v2v much more, into
more streamlined tools (but without breaking backwards compatibility
or removing features).  Still don't know how, but I'll read the rest
of your email in more detail as part of thinking about this.

At the moment I'm working on separating virt-v2v from the huge
libguestfs repository, which I think is the first step to really
refactoring it properly.

Thanks,

Rich.

> 
> Such a tool (let's call it, say, virt-adopt or virt-adapt, depending on
> whose POV you take, the guest's or the host's :) could have a number of
> benefits:
> 
> 1) it can be used outside of the current virt-v2v, e.g.
> 
>    a) v2v where the rest of migration is taken care of by something
>       else.  E.g. we use it to migrate from our own older releases that
>       used a different hypervisor, and we're sure to do a better job
>       converting configurations and copying data than the generic tool.
>       Overall data/metadata migration is the place where plenty of
>       customization can be necessary, and the monolithic tool is too
>       rigid for that, so we've had a few other cases where it was easier
>       to write custom migration code and call the tool to adopt the
>       resulting guest.  In particular, QCOW2 overlays are not always
>       possible or conveninent, data migration via the source hypervisor
>       control url is not necessarily the fastest, and so on.
> 
>    b) we use it to restore backups created with the older releases which
>       used incompatible virtual hardware.
> 
>    c) every now and then I wanted to change the hardware configuration
>       of an existing VM in an incompatible way, like changing the
>       chipset, the type of the storage or network controller, etc, and
>       this tool came in handy.
> 
> 2) it can allow splitting the current virt-v2v into reusable steps.
>    E.g. it's a pity, having migrated a huge guest disk image, to find
>    out that due to some minor network misconfiguration on the host you
>    can't run yum in the guestfs appliance to install some missing bits
>    and thus can't complete the conversion, so, once the problem is
>    fixed, you have to start over.
> 
> 3) it fits better with the libguestfs' purpose, which is to poke in the
>    guest's files.  As a result, in particular, it has smaller testing
>    scope, and is thus easier to keep tested.  (I wonder how many
>    contributors have the opportunity to test all virt-v2v's input and
>    output hypervisor combinations.)
> 
> The only downside I see is that it requires some work to be done, and
> I'm short of spare cycles to do that :(
> 
> > What you want -- copy done elsewhere, convert in place, create metadata --
> > is a very different flow.  It could look something like this:
> > 
> > * let overlays = create_overlays source.s_disks in
> >   let g = Guestfs.guestfs () in
> >   populate_overlays g overlays
> >   let inspect = Inspect_source.inspect_source g in
> >   let guestcaps = do_convert g inspect output in
> >   let targets = output#prepare_targets overlays in
> >   for each overlay: qemu-img commit it
> >   output#create_metadata source targets guestcaps inspect
> 
> FWIW our scenarios look somewhat different:
> 
> 1) transfer vm disks and metadata
> 2) (optional) inspect migrated guest
> 3) create target vm based on (1) and (2)
> 4) (optional) snapshot target vm (using overlays or whatever is appropriate)
> 5) "adopt" target vm (using virt-v2v --in-place)
> 6) (optional) boot target vm and let firstboot scripts run
> 7) (optional) merge snapshots with the original disks
> 8) release target vm to user
> 
> Note, in particular, that step (5) makes no decisions re. the eventual
> vm configuration; they are all made at step (3).  Restore from old
> backups just changes step (1).
> 
> Thanks, and sorry for the long mail without patches,
> Roman.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
Fedora Windows cross-compiler. Compile Windows programs, test, and
build Windows installers. Over 100 libraries supported.
http://fedoraproject.org/wiki/MinGW