[Libvir] Thoughts on remote storage support

[Richard W.M. Jones]

Tóth István wrote:
> Richard W.M. Jones wrote:
>> For example, here are the domains and their images running on my Xen 
>> host at the moment.  I got this by writing a simple script which 
>> parses the domain XML:
>>
>> fc6_0:
>>         /var/lib/xen/images/fc6_0.img -> xvda
>>         /var/lib/xen/images/home.disk -> xvdb
>> fc6_1:
>>         /var/lib/xen/images/fc6_1.img -> xvda
>> debian32fv:
>>         /var/lib/xen/images/debian32fv.img -> hda
>> f764pv:
>>         /dev/Images/f764pv -> xvda
>> freebsd32fv:
>>         /var/lib/xen/images/freebsd32fv.img -> hda
>>         [CD] -> hdc
>> gentoo32fv:
>>         /var/lib/xen/images/gentoo32fv.img -> hda
> Well, this is a good handle for the images that belong too an active 
> domain.
> But I can see other images laying around, backup images, snapshots, 
> virgin installed images for provisioning of new VMs, and you need to 
> refer to those as well.
> Hence, I still think that it would be better to use host+path. For 
> example, you need to be able to say in effect: copy 
> "/var/lib/xen/images/fc6_0.img" to "/backups/fc6_xvda_1", and you have 
> to refer to target image somehow.
> You could just use the local path in this case, but I think that being 
> able work with images on other libvirt hosts would be a bonus.

There's an open-ended access control problem here.  libvirtd runs as 
root and host+path gives a way to read and write any file on the system.

Better might be to allow the system administrator to configure 
directories where backup images, snapshots and so on may be located 
(through /etc/libvirtd.conf), and have libvirtd check this, and also 
have an additional level of enforcement through SELinux (as is done with 
Xen images now).

For my rather limited needs with virt-df I was going to propose an API 
like this:

   virDomainPeekDevice (virDomainPtr domain,
		       const char *path,
		       off_t offset, off_t size,
		       char *result_buffer);

The security check would be something along these lines:

  * path must be a source device (as returned in the domain XML)
  * path must belong to the domain
  * offset, size must be entirely within the path device/file

This check could be extended to allow path to be in the configured 
backup / snapshot directories.

(This is not really thought through at the moment, however comments 
welcome).

With that call we then need to look at "virtualising" libparted so that 
instead of making direct read(2), lseek(2) etc. system calls, these may 
be redirected through a VFS layer which would call virDomainPeekDevice.

(I'm sure I posted something about this to the list, but that was two 
weeks ago, I've been on holiday, I'm jetlagged, and now I can't find it...)

[...]
> In fact, the more I thought about it, and the more scenarios popped into 
> my mind (plus the ones you describe above), the more I think that at 
> least an initial implementation should not try to see into the 
> partition, exactly because of the problems you mention above.
> Even if partition/filesystem handling is included in libvirt, it should 
> probably be somewhat orthogonal to the rest of the image handling 
> functions.
> 
> i.e. the operations I detailed below (except for the growfs-related 
> ones) for creating, moving, backing up, etc. of raw images, and a 
> different set of operations that partitions, adds/removes paritions, 
> creates file systems, growfs-es, etc.
> 
> This limits the complexity to just supporting simple files, block 
> devices, and LVMs ( or the equivalent functionality on other platforms), 
> and the parted-like functionality can be added on top of it.

My thinking about this moved along a bit: What if we explicitly _don't_ 
think about supporting LVM operations and so on within libvirt.  Making 
a general-purpose solution is a big, intractable problem.

Instead we could allow the system administrator to create some 
operations (again, through /etc/libvirtd.conf [1]):

----- /etc/libvirtd.conf -------------------

allocate partition: "lvcreate -L %size -n %name XenVolGroup"
list partitions: "lvs"

--------------------------------------------

On the libvirt side those turn into standard calls like:

   virConnectListPartitions (...)

If the commands don't exist in libvirtd.conf then those calls fail with 
a suitable error message.

We can set up suggested commands in the default configuration for Linux 
+ LVM, Linux + partitions, Solaris, etc. but it defers the policy to 
system administrators.

Rich.

[1] libvirtd.conf is only available in the remote case, so perhaps we 
need also a libvirt.conf for the local case.

-- 
Emerging Technologies, Red Hat - http://et.redhat.com/~rjones/
Registered Address: Red Hat UK Ltd, Amberley Place, 107-111 Peascod
Street, Windsor, Berkshire, SL4 1TE, United Kingdom.  Registered in
England and Wales under Company Registration No. 03798903

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3237 bytes
Desc: S/MIME Cryptographic Signature
URL: <http://listman.redhat.com/archives/libvir-list/attachments/20071015/2fe445f0/attachment-0001.bin>