[libvirt] Qemu/KVM is 3x slower under libvirt

[Daniel P. Berrange]

On Wed, Sep 28, 2011 at 11:19:43AM +0200, Reeted wrote:
> On 09/28/11 09:51, Daniel P. Berrange wrote:
> >>This is my bash commandline:
> >>
> >>/opt/qemu-kvm-0.14.1/bin/qemu-system-x86_64 -M pc-0.14 -enable-kvm
> >>-m 2002 -smp 2,sockets=2,cores=1,threads=1 -name vmname1-1 -uuid
> >>ee75e28a-3bf3-78d9-3cba-65aa63973380 -nodefconfig -nodefaults
> >>-chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/vmname1-1.monitor,server,nowait
> >>-mon chardev=charmonitor,id=monitor,mode=readline -rtc base=utc
> >>-boot order=dc,menu=on -drive file=/dev/mapper/vgPtpVM-lvVM_Vmname1_d1,if=none,id=drive-virtio-disk0,boot=on,format=raw,cache=none,aio=native
> >>-device virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=virtio-disk0
> >>-drive if=none,media=cdrom,id=drive-ide0-1-0,readonly=on,format=raw,cache=none,aio=native
> >>-device ide-drive,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0
> >>-net nic,model=virtio -net tap,ifname=tap0,script=no,downscript=no
> >>-usb -vnc 127.0.0.1:0 -vga cirrus -device
> >>virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x5
> >
> >This shows KVM is being requested, but we should validate that KVM is
> >definitely being activated when under libvirt. You can test this by
> >doing:
> >
> >     virsh qemu-monitor-command vmname1 'info kvm'
> 
> kvm support: enabled
> 
> I think I would see a higher impact if it was KVM not enabled.
> 
> >>Which was taken from libvirt's command line. The only modifications
> >>I did to the original libvirt commandline (seen with ps aux) were:


> >>- Network was: -netdev tap,fd=17,id=hostnet0,vhost=on,vhostfd=18
> >>-device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:05:36:60,bus=pci.0,addr=0x3
> >>Has been simplified to: -net nic,model=virtio -net
> >>tap,ifname=tap0,script=no,downscript=no
> >>and manual bridging of the tap0 interface.
> >You could have equivalently used
> >
> >  -netdev tap,ifname=tap0,script=no,downscript=no,id=hostnet0,vhost=on
> >  -device virtio-net-pci,netdev=hostnet0,id=net0,mac=52:54:00:05:36:60,bus=pci.0,addr=0x3
> 
> It's this! It's this!! (thanks for the line)
> 
> It raises boot time by 10-13 seconds

Ok, that is truely bizarre and I don't really have any explanation
for why that is. I guess you could try 'vhost=off' too and see if that
makes the difference.

> 
> But now I don't know where to look.... During boot there is a pause
> usually between /scripts/init-bottom  (Ubuntu 11.04 guest) and the
> appearance of login prompt, however that is not really meaningful
> because there is probably much background activity going on there,
> with init etc. which don't display messages
> 
> 
> init-bottom does just this
> 
> ---------------------------------
> #!/bin/sh -e
> # initramfs init-bottom script for udev
> 
> PREREQ=""
> 
> # Output pre-requisites
> prereqs()
> {
>         echo "$PREREQ"
> }
> 
> case "$1" in
>     prereqs)
>         prereqs
>         exit 0
>         ;;
> esac
> 
> 
> # Stop udevd, we'll miss a few events while we run init, but we catch up
> pkill udevd
> 
> # Move /dev to the real filesystem
> mount -n -o move /dev ${rootmnt}/dev
> ---------------------------------
> 
> It doesn't look like it should take time to execute.
> So there is probably some other background activity going on... and
> that is slower, but I don't know what that is.
> 
> 
> Another thing that can be noticed is that the dmesg message:
> 
> [   13.290173] eth0: no IPv6 routers present
> 
> (which is also the last message)
> 
> happens on average 1 (one) second earlier in the fast case (-net)
> than in the slow case (-netdev)

Hmm, none of that looks particularly suspect. So I don't really have
much idea what else to try apart from the 'vhost=off' possibilty.


Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|