[libvirt] libguestfs integration: rich disk access for libvirt applications

[Richard W.M. Jones]

On Tue, Sep 27, 2011 at 10:10:00AM +0100, Stefan Hajnoczi wrote:
> Libguestfs provides very nice functionality for applications that need
> to work with disk images.  The includes provisioning applications that
> set up or customize disk images.  It also includes backup applications
> that want to look inside disk image snapshots - both at the block and
> file level.
> 
> What's missing for libguestfs to fill this role is integration that
> allows libvirt and libguestfs to work together with the same
> network-transparent remote API approach.
> 
> In the past we have discussed remoting libguestfs and Richard
> presented possible approaches:
> https://www.redhat.com/archives/libguestfs/2011-May/msg00093.html
> 
> Could libvirt could provide a secure channel over which the libguestfs
> client that an application is linked against talks to the remote
> guestfsd?
> 
> Would libguestfs need some new parameters to connect to a remote
> libvirtd and create its appliance VM?
> 
> In terms of application use cases, I'm thinking along the lines of
> using libvirt to enumerate storage volumes and then switching to
> libguestfs to actually access the storage volume remotely.  Is there
> enough information exposed by libvirt today to switch over to
> libguestfs and point it at the storage volume/image file?

I know that Dan had some ideas in this area, so thanks for CC-ing him.

Ideally what we'd like to have happen is that the existing tools just
work for remote domains, eg:

  virt-edit -d Guest /etc/motd

  virt-edit -c qemu+ssh://remote/system -d Guest /etc/motd

would both just work without the user needing to do anything special.
How to make that happen is the complicated bit ...

Currently what the '-d' option ('guestfs_add_domain' API) does is it
grabs the guest libvirt XML, parses out the <disk> element, and looks
for the path to the disk.  We then just open this path as a local
file.  In the remote case, one of three things can happen:

 - path doesn't exist so we fail (the usual case)

 - path does exist because the user has arranged for paths to be
   available on the local machine (think: SAN)

 - we open some other unrelated disk image (oops)

Libvirt isn't giving us any indication that the guest is located
remotely.

What would be nice would be that libvirtd [which is on the same host
as the guest always] opens the disk image for us and by some method
forwards this back to us.  Ideally only in the remote case, so that we
can work as efficiently as possible in the local case.

I can see several ways this could work:

(1) We use the existing virDomainBlockPeek API.  libvirt already has
an API for peeking into a remote disk.  It's inefficient and is
limited to max 64K read per message, and writing is not possible.
qemu would need to be modified to be able to access this API (or we
could write some sort of libvirt<->nbd translation layer).

  http://libvirt.org/html/libvirt-libvirt.html#virDomainBlockPeek

(2) libvirtd runs nbd-server and forwards this back to us.  How would
qemu access this on the client side?

(3) libvirtd runs the libguestfs appliance.  It forwards the socket
back to us over the libvirt connection.  I think this is what Dan had
in mind, and it's reasonably easy to integrate this into both libvirtd
and libguestfs.

(4) libvirt somehow gains the whole libguestfs API and forwards API
calls.

Others ways ...?

What's the next step to make this happen?

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines.  Supports shell scripting,
bindings from many languages.  http://libguestfs.org