[libvirt] [PATCH 1/1] qemu: add support for multiple gluster hosts

[Deepak C Shetty]

On 10/05/2015 07:33 PM, Peter Krempa wrote:
> On Mon, Oct 05, 2015 at 18:59:54 +0530, Deepak C Shetty wrote:
> [Note: Formatting of this mail is broken. Please configure your mail
> client to properly flow text in the plaintext mode.]

Sorry about that. I configured plain text format for libvir-list at redhat.com
domain in my thunderbird client and selected plain text as delivery format
for this specificmail.

>
>> On 10/05/2015 04:31 PM, Peter Krempa wrote:
>>> On Mon, Oct 05, 2015 at 15:30:42 +0530, Prasanna Kumar Kalever wrote:
> [snip]
>
>>> So while all the above looks pretty okay at this point (except for
>>> clearing the image type) this patch is incomplete.
>>>
>>> Since libvirt is loading the backing chain to be able to traverse it and
>>> set correct permissions on individual images. This equals to two
>>> additional parts of the code that need modification:
>>>
>>> 1) virStorageFileBackendGlusterInit has the same "src->nhosts != 1"
>>> condition and is not prepared to work with multiple protocols
>> IIUC this is part of the storage pool feature of libvirt and that can be
>> handled
>> as part of a separate patch. Using gluster as a block backend for
>> qemudoesn't
>> enforce using gluster as a file backend for libvirt storage pool, they
>> both are
>> separate, IMHO
> No, the part of the storage driver I'm refering to is used in the
> backing chain detection. So even if you don't use the libvirt's gluster
> storage pool, the code I'm refering to will be invoked.
>
> It can be a separate patch, but it needs to go in before this patch
> lands.
>
>>> 2) I presume (I didn't test it yet) that the qemu patch that adds this
>>> stuff will result into using the "json:{" protocol string in the
>>> 'backing_store' field once you do a snapshot of a gluster disk that uses
>>> this new syntax. If that happens the VM will not be able to start in
>>> libvirt any more, as libvirt does not have a 'json:{' protocol parser to
>>> parse the backing store.
>> We tried to take a disk-only external snapshot with libvirtd running with
>> this patch and I was unable to see the "json:{..." format for backing file.
>>
>> I think we understand what you are saying, but we are unable to reproduce
>> the problem ( of having the json way in backing file). This is what we
>> did ....
>>
>> * Domain HFM running with CentOs7.qcow2 as a network disk type w/
>> protocol=gluster
>>
>> 0 :) dhcp1-86 ~/deepakcs $ virsh list --all
>>    Id    Name                           State
>> ----------------------------------------------------
>>    4     HFM                            running
>>
>>
>> * Create a qcow2 file for use with external snapshot
>>
>> qemu-img create -f qcow2 -b gluster://10.70.1.86/sample/CentOs7.qcow2
>> gluster://10.70.1.86/sample/newsnap.qcow2
> If you pre-create backing files to use with --reuse-external, you have
> to use the json syntax here to express the backing file.

We tried that, but qemu-img says invalid protocol/format when given a
json:"{...}" string in the above cmd, for backing and new file names.

Qemu being used is built from source using the qemu patch thats
referenced with the link below, that has json support for gluster network
storage.


>
>> * Create a domainsnapshot XML (snap.xml)
>>
>> <domainsnapshot>
>>       <disks>
>>           <disk name="vda" snapshot="external" type="network">
>>               <source protocol="gluster" name="sample/newsnap.qcow2">
>>                   <host name="10.70.1.86" port="24007"/>
>>                   <host name="10.70.1.88" port="24007"/>
>>                   <host name="10.70.1.87" port="24007"/>
>>               </source>
>>           </disk>
>>       </disks>
>> </domainsnapshot>
> You are doing it the other way around. I was speaking of doing a VM
> started with a multi-server gluster volume and then doing a snapshot
> wherever you like. This should result in a snapshot that has json
> definition of the backing file.

Not clear what you mean by 'other way around' ?

A VM was created (HFM seen above) using libvirt with this patch. The HFM's
domain xml disk element is of type network & has 3 host's (hence 
multi-server) entries.

Also not clear on what you mean by 'whereeve u like' :) Can you pls 
elaborate or provide
the high level virsh steps that we need to use ?

We are using virsh which is again built as part of the libvirt src build 
with this patch

>
>> * Run virsh snap cmd (error in bold)
> [Note: using HTML mails on technical lists is not a good idea.]
Sorry again, this mail should be in plain text

>
>> 0 :) dhcp1-86 ~/deepakcs $ virsh snapshot-create HFM --xmlfile
>> ./snap.xml --disk-only --reuse-external
>> *error: internal error: missing storage backend for network files using
>> gluster protocol*
> Erm, how are you attempting to test this if you don't even have libvirt
> with gluster support installed?

Hmm, what makes u think we don't have libvirt with gluster support 
installed ?
libvirtd is running from source with this patch included.

Are we missing anything ?

>
>> * Run virsh snap cmd w/ domain is off (error in bold)
>>
>> 0 :) dhcp1-86 ~/deepakcs $  virsh list --all
>>    Id    Name                           State
>> ----------------------------------------------------
>>    -     HFM                            shut off
>>
>> 0 :) dhcp1-86 ~/deepakcs $ virsh snapshot-create HFM --xmlfile
>> ./snap.xml --disk-only --reuse-external
>> *error: internal error: external inactive snapshots are not supported on
>> 'network' disks using 'gluster' protocol*
> This is expected, inactive external snapshots have only a very limited
> set of supported configurations. Networked storage is not there.
>
>>> So in order to accept this patch, the helpers in
>>> src/util/virstoragefile.c (virStorageSourceParseBackingURI) will need to
>>> be made modular so that they can accept a driver specific callback, that
>>> will then parse the 'json:{' source definitions. (We can't just hardcode
>>> it there, since src/util "should" not contain any qemu specific code.)
>> Questions:
>>       1) Why does snapshot doesn't work, is it bcos the support for json
>> is not present in the backup parse code ?
>>          The error msg seen above, isn't clear
> Well the error message says that you have libvirt without gluster
> support and thus it refuses to do a snapshot on a gluster disk.
>
>>       2) IIUC, you mean that the backing file will be in the json format
>> something like ...
>>          backing file: json"{...}" in `qemu-img info --backing-chain`
>> output ?
> Exactly.
>
>>         How to create such a scenario, so that we can "see" the problem
>> happening and hence debug/fix it :)
> Start a VM with the multi-host gluster support (as of by this patch)
> and create an external snapshot even on local disk, no need to go to
> gluster again.
HFM is running w/ libvirtd and qemu buitl from sources with the 
respective patches included

Can you pls elaborate on "even on local disk" ?

thanx,
deepak