[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: [virt-tools-list] virt-v2v question involving esx

On 18/03/11 13:04, Richard W.M. Jones wrote:
On Fri, Mar 18, 2011 at 08:40:31AM -0400, Kenneth Armstrong wrote:
On Fri, Mar 18, 2011 at 8:06 AM, Dennis Jacobfeuerborn
<dennisml conversis de>  wrote:
On 03/18/2011 11:34 AM, Matthew Booth wrote:

On 17/03/11 19:27, Richard W.M. Jones wrote:

[Adding Matt]

On Wed, Mar 16, 2011 at 02:59:40PM -0400, Kenneth Armstrong wrote:

I have a vm on an esx server that I want to convert using virt-v2v. I
know that it can connect to the esx server and copy the vmdk files
down to do its conversion, but can I just use wget or something like
that to copy the vmdk files down first, then use virt-v2v to convert
the vm?

The reason that I ask is that it would be better for me to do it this
way, so that after I copy the vm files down, I can start the vm back
up and let it be used while virt-v2v does the conversion on the copied
image files. I like virt-v2v, but it takes a long time to do it's job
with the copying.

Are you saying that wget is substantially faster than virt-v2v's copying?
ask because virt-v2v uses the same http interface to retrieve the image.
actually streams and writes directly to the target destination, so unless
the target destination is slow, I wouldn't expect this to save you a
significant amount of time. The actual conversion process usually takes
about a minute, so the copying time dominates almost completely.

It is possible to do what you're asking, but it's a hassle. You'd need to
obtain the storage files, create a local libvirt XML file which points to
the local copies, then run virt-v2v using the libvirtxml input method.

If the vmdk file is sparse then you might end up copying a lot less data
compared to a virt-v2v copy and since as you point out the copying is the
thing that dominates the migration time that can potentially be a big win.
It would be nice if virt-v2v had an option to not copy the block device but
instead the contents of a filesystem but that would probably make things
much more complicated since you'd have to support all kinds of filesystems,
LVM, potentially also copy UUID's etc., etc.

Dennis is spot on.

If I copy down the file straight from the ESX server's web interface,
it's quite quick.  However, when I try it through virt-v2v, the
process takes hours downloading/converting the same file.

The vmdk file in question is a sparse disk.  So I assume that virt-v2v
is expanding the disk out during the whole process?


Sparseness isn't doing anything for you here. HTTP doesn't have the ability to represent sparseness[1], so it's copying all the zeroes 1 by 1 in both cases. In order to get any benefit from sparseness in this case you'd need filesystem level access. I considered sshfs for this, but unfortunately it can't support the ioctl required to efficiently detect sparseness on the remote end. This means you're still left copying over all the data in order to find sparse sections. The only remaining advantage to sshfs is that it supports compression, which obviously works well on sparse files.

To summarise, in order to exploit sparseness on ESX we'd need either:

1. An agent running on the ESX server which can exploit sparseness.
2. An explicitly sparse file format.

Unless the former already exists (e.g. sftp-server) and I've missed it, this approach likely won't fly, as it would require the ESX administrator to install non-VMware software on their server.

I believe vmdk is able to do explicit sparse, although in practise it never seems to be configured this way. Even if the guest was configured this way, qemu doesn't support it, so we can't use it yet.

So this comes down to: why is virt-v2v's copy so much slower than wget doing the same thing? It could be down to inefficient chunking or similar, but I'd guess it's more likely related to where it's writing to. Is it writing to NFS by any chance? If so, what's the network like between you and the NFS target?


[1] Unless it supports compression, which nothing does in practise.
Matthew Booth, RHCA, RHCSS
Red Hat Engineering, Virtualisation Team

GPG ID:  D33C3490
GPG FPR: 3733 612D 2D05 5458 8A8A 1600 3441 EA19 D33C 3490

[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]