[fedora-virt] Fedora virt status
Mark McLoughlin
markmc at redhat.com
Fri Aug 21 11:55:08 UTC 2009
It's been a busy seven weeks or so since I sent the last one of
these. I'll try not to leave such a big gap between status reports in
future! :-)
F-12 Schedule
=============
The Fedora 12 Alpha release is now baked and will be released next
week on August 25th.
The next big deadline coming up is the Final Development freeze on
September 29th. After that date, only important bug fixes will be
accepted.
http://fedoraproject.org/wiki/Releases/12/Schedule
F-12 Features
=============
The final list of virt features for Fedora 12 looks like:
* libguestfs
* KSM
* KVM Huge Page Backed Memory
* KVM NIC Hotplug
* KVM qcow2 Performance
* KVM Stable Guest ABI
* Network Interface Management
* SR-IOV
* VirtgPXE
* VirtPrivileges
* VirtTCK
* VirtStorageManagement
(Note, FESCo didn't approve TCK as a feature, but that should't stop
us pimping it :-)
See:
https://fedoraproject.org/wiki/Category:F12_Virt_Features
F-12 Changes to System Defaults
===============================
There are a couple of changes to Fedora 12 system defaults that are
related to virtualization:
https://fedoraproject.org/wiki/Fedora_12_Alpha_release_notes#Changes_to_System_Defaults
For security and performance reasons, iptables rules are no longer
applied by default to frames forwarded across linux kernel ethernet
bridges. See bug #512206 for more details on the rationale behind
this change.
Historically, uids and gids 0-100 are reserved for specific system
accounts and allocated via the uidgid file in the setup
package. This space has now been exhausted and 0-200 is now
reserved. This should not be an issue on most systems because
dynamically allocated system accounts are usually allocated
downwards from 499. See bug 515779 and bug #511957 for more details.
F-13 Features
=============
We already have quite a number of features planned for Fedora 13. See:
https://fedoraproject.org/wiki/Category:F12_Virt_Features
One of the most interesting of those is Michael Tsirkin's "kernel
acceleration for KVM networking":
https://fedoraproject.org/wiki/Features/VHostNet
The idea is to add a kernel module which much more efficiently takes
care of the packet handling part of the virtio_net host backend. The
progress of this feature can be followed on the Linux Foundation
virtualization mailing list:
https://lists.linux-foundation.org/pipermail/virtualization/2009-August/date.html
New Releases
============
Several new releases of various virt bits have been released recently:
- qemu-kvm-0.10.6:
http://www.mail-archive.com/kvm@vger.kernel.org/msg20161.html
- qemu-kvm-0.11.0-rc1:
http://www.mail-archive.com/kvm@vger.kernel.org/msg20168.html
- libvirt-0.7.0:
http://www.redhat.com/archives/libvir-list/2009-August/msg00080.html
- python-virtinst-0.500.0:
http://www.redhat.com/archives/virt-tools-list/2009-July/msg00055.html
- virt-manager-0.8.0:
http://www.redhat.com/archives/virt-tools-list/2009-July/msg00054.html
- libvirt-java-0.3.0:
http://www.redhat.com/archives/libvir-list/2009-July/msg01011.html
- libguestfs-1.0.67:
https://www.redhat.com/archives/libguestfs/2009-August/msg00281.html
Fedora Weekly News
==================
Unlike me, Dale Bewley is no slacker and has kept the FWN updates
coming:
https://fedoraproject.org/wiki/FWN/Issue184#Virtualization
https://fedoraproject.org/wiki/FWN/Issue185#Virtualization
https://fedoraproject.org/wiki/FWN/Issue186#Virtualization
https://fedoraproject.org/wiki/FWN/Issue187#Virtualization
https://fedoraproject.org/wiki/FWN/Issue189#Virtualization
libguestfs
==========
Rich Jones announced that libguestfs has its own (very busy) mailing
list now:
http://www.redhat.com/mailman/listinfo/libguestfs
Also of note is that the virt-df utility has now been re-written to
use libguestfs.
virt-tools-list
===============
The badly named et-mgmt-tools mailing list has been deprecated in
favour of a new virt-tools-list:
http://www.redhat.com/archives/virt-tools-list/2009-July/msg00001.html
This list originally came into being as a place for discussing
projects under Red Hat's 'emerging technology' moniker, hence the
prefix 'et-'. In retrospect this was a really bad choice of names
for a mailing list and causes endless confusion for people wrt what
to discuss where. Most of the emerging technology projects have
lists of their own (cobbler, augeas, libguestfs, libvirt) and it is
about time that virt-manager and friends joined them.
To that end we have created a new mailing list
'virt-tools-list'. This will be the new home for all developer &
user discussions relating to the following applications
- virt-manager
- virt-viewer
- virt-install
- virt-clone
- virt-image
- virt-convert
Xen
===
M A Young, Pasi Kärkkäinen and others are continuing to work hard
testing builds of latest upstream pv_ops Dom0. See the fedora-xen
mailing list archives:
http://www.redhat.com/archives/fedora-xen/2009-August/date.html
Gerd Hoffman has updated Fedora 12 to xen-3.4.1:
http://www.redhat.com/archives/fedora-virt/2009-August/msg00036.html
Fedora 12's Xen DomU support has seen a number of problems.
https://bugzilla.redhat.com/515831
Switch bzImage from LZMA back to gzip compression so Xen can load
Fedora kernels again
It turns out that Fedora switched their bzImage format from
gzip to LZMA, which the Xen loader doesn't support. This has been
reverted until Fedora 13, giving Xen a chance to catch up.
Chris Lalancette quickly took on the task of making sure that we have
LZMA support in the Xen domain builder. Patches for this are upstream
now and we just need them pulled into Fedora 12:
https://bugzilla.redhat.com/518588
Add xen domain builder support for bzImage lzma/bzip2 compression
However, we're not done yet. The F12 kernel still doesn't boot as a
DomU:
https://bugzilla.redhat.com/508120
2.6.31-rc1 xen domU crashes early during boot
It now turns out that the F12 kernel crashes during boot in Xen
DomU. Jeremy Fitzhardinge has come up with patches to fix at least
some of this, but it sounds like there are more dragons lurking
here.
Michael Schmidt points out this xenfb issue:
http://lkml.org/lkml/2009/8/21/71
...
So it crashes during Xen framebuffer initialization. And indeed,
disabling CONFIG_XEN_FBDEV_FRONTEND helps, the kernel then boots
fine.
...
Fedora QEMU/KVM Security
========================
There are several things to bear in mind wrt to libvirt's support for
qemu/kvm and security:
1) The qemu process now runs as the qemu user, not root. This
reduces the ability of the process to attack the host if it is
compromised. However, users should be aware of the potential for
issues with e.g. directories having the wrong permissions.
2) qemu processes are also confined using SELinux sVirt
protection. This reduces the ability of the process to attack
other qemu processes if it is compromised. Again, though, there
is the potential for users to see problems caused by e.g. files
not being labelled correctly.
Dan Berrange prepared a comprehensive set of docs on the security
architecture for libvirt's qemu driver:
http://libvirt.org/drvqemu.html
Some of the recently active bug reports in this are include:
https://bugzilla.redhat.com/515671
'groupadd -r' allocates gids upwards
https://bugzilla.redhat.com/515667
login.defs/SYS_UID_MIN should be 200
The qemu uidgid reservation is 107, but 'useradd/groupadd -r' are
still allocating out of the 100-500 range. It wasn't such a big
problem when they used to allocate downwards from the top of the
system accounts range, but this behaviour changed recently.
https://bugzilla.redhat.com/497341
Make the /dev/kvm device world accessible to all users by default
https://bugzilla.redhat.com/346151
Create a kvm user account and kvm group
https://bugzilla.redhat.com/500472
QEMU driver should run all QEMU VMs as non-root system account
All done by danpb for F-12 as part of the VirtPrivileges
feature.
https://bugzilla.redhat.com/507397
Directory permissions on volume group directory too restrictive
The VirtPrivileges feature requires that the LVM volume group
directory permissions are relaxed a bit. Apparently this should be
magically fixed by lvm using udev but, although it has switched to
udev now, it doesn't seem to have changed anything.
https://bugzilla.redhat.com/515547
libvirt fails to start guest - Failed to set security label
An selinux-policy regression in Fedora 12 caused libvirt to
break. Fixed in rawhide now.
https://bugzilla.redhat.com/515521
SELinux is preventing qemu-kvm (svirt_t) "setrlimit" svirt_t
An SELinux setrlimit() denial is causing qemu to fail to start for
some F-11 users. At first, we had no idea where setrlimit() is
being called from but Jerry James figured out that it was glibc.
It turns out that glibc has a workaround for the fact that
/dev/pts was incorrectly mounted in F-11 and an selinux-policy
update to allow glibc to run that workaround has now been pushed.
It also turns out that qemu isn't setting some file descriptors to
CLOEXEC and this is causing selinux problems when pt_chown is
exec()ed.
https://bugzilla.redhat.com/518014
Allow svirt images to create sock_file in svirt_var_run_t
A Fedora 11 selinux-policy update needed to use the virt-preview
version of libvirt.
https://bugzilla.redhat.com/496442
libvirt only relabels disks *after* hotplugging them into QEMU
A fix for this issue has been backported to F-11. It fixes
problems like not being able to attach a dvd/cdrom to a guest in
virt-manager.
https://bugzilla.redhat.com/516430
libvirt cannot re-label a disk image under an NTFS partition
Because NTFS doesn't support xattrs, svirt cannot start a guest
with disk images on an NTFS partition.
https://bugzilla.redhat.com/516034
libvirt is not chowning kernel/initrd images before launching qemu
As part of the F-12 VirtPrivileges feature we started running the
qemu process unprivileged, but we neglected to chown kernel and
initrd images before launching qemu. Fixed now in F-12 Alpha.
https://bugzilla.redhat.com/517157
libvirt fails to start guest on NFS even when sebool virt_use_nfs
is on
David Lutterkort notes that libvirt is defeating the purpose of
the virt_use_nfs sebool by refusing to start a guest if it can't
relabel its disk images.
https://bugzilla.redhat.com/517304
libvirt needs to better handle chown-ing images on NFS shares
Now that we're chown-ing images before starting guests, we need to
make various improvements in order to handle NFS shares.
https://bugzilla.redhat.com/517617
libvirt/netcf loads modprobe.conf and others - AVC messages
(preventing libvirtd (virtd_t) "getattr" modules_conf_t)
libvirt's new network interface configuration support
(unsupringly) touches a bunch of files in /etc, so we need policy
updates to allow libvirtd to do that.
https://bugzilla.redhat.com/517619
libvirt fails to start guest with qemu configured to run as
root/root
There seems to be a selinux-policy issue where if libvirt is
configured to run guests as root/root, they fail to transition to
svirt_t. Strangely, the AVCs persist when you change the
configuration back until you reboot, even though the transitions
do appear to be succeeding.
Aside from the AVCs, we need to make libvirt chown various
directories to the user is going to run qemu as.
https://bugzilla.redhat.com/517379
virt-manager should warn if guest images will are not readable by
qemu
If a user downloads an ISO to her homedir and tries to start a
guest using it, it fails because qemu doesn't have permissions to
the homedir. We could warn the user of this common scenario.
KVM PCI Device Assignment
=========================
A number of improvements to the feature introduced in Fedora 11 are
now available as an update:
https://bugzilla.redhat.com/515689
libvirt should allow PCI PM reset on multi-function devices
https://bugzilla.redhat.com/499561
libvirt does not automatically re-attach an assigned device in the
host after guest shutdown
https://bugzilla.redhat.com/499678
libvirt should be able to reset a PCI function even if it causes
other unused devices/functions to be reset
https://bugzilla.redhat.com/515689
libvirt should allow PCI PM reset on multi-function devices
Also, tieing in with the recent work to add KVM NIC hotplug support to
libvirt, we now have support in Fedora 12 for assigned device hotplug:
https://bugzilla.redhat.com/517464
Add support to libvirt for KVM PCI device assignment hotplug
Bugs
====
The last while has seen a huge churn of bugs in bugzilla, leaving us
with a DOOM-O-METER of 217 now. Seven weeks ago we were up to 250.
If you're looking to help getting this number down even further, the
place to start is the Fedora 12 blocker and target lists:
https://bugzilla.redhat.com/showdependencytree.cgi?id=F12VirtBlocker&hide_resolved=1
https://bugzilla.redhat.com/showdependencytree.cgi?id=F12VirtTarget&hide_resolved=1
Ongoing Bugs
============
== misc ==
https://bugzilla.redhat.com/509702
Implement support for CLONE_IO
Request for glibc to support CLONE_IO. Uli suggests that CLONE_IO
should be used by default. Avi suggests that it shouldn't.
== kernel ==
https://bugzilla.redhat.com/509383
rotational mode is much faster for virtio-blk disks, but uses
non-rotational mode by default
This issue is still ongoing, we need to get the default changed.
https://bugzilla.redhat.com/512358
Unable to boot using qemu-kvm and gPXE from virt-preview
repository
We need a backport of a kvm.ko fix in order to be able to use gPXE
on an F-11 host.
https://bugzilla.redhat.com/515741
2.6.30 kernel stopped supporting xattrs on hugetlbfs
This issue is preventing libvirt from using SELinux labels to
enforce separation between qemu guests using huge page backed
memory. John Cooper is working to fix this for the KVM Huge Page
Backed Memory feature in Fedora 12.
https://bugzilla.redhat.com/516909
KSM breaks encryption 157 > kernel > 139 - KSM support now
disabled
A recent set of KSM changes from upstream has caused a regression
with encrypted volumes. KSM has been disabled until this is
fixed.
https://bugzilla.redhat.com/518022
2.6.31 virtio_net oops in skb_copy_from_linear_data_offset()
James Laska hit this nice oops during an F12 guest install over
NFS.
== qemu ==
https://bugzilla.redhat.com/477955
Enable qemu sound devices to tunnel over VNC
https://bugzilla.redhat.com/508317
Allow sounds devices to be used with svirt - tunnel sound over VNC
These bugs have been moved to F13VirtTarget now that the feature
has been punted to Fedora 13.
https://bugzilla.redhat.com/512376
Guest clock is running aprox. 3 seconds before host clock
Strange problem with the guest clock consistently being a few
seconds behind the host clock. Removing hwclock from the system
reduces the offset to below one second. This is beginning to look
like a fundamental problem with the rtc resolution and using
hwclock to sync the system time during boot. Glauber proposes
removing 88-clock.rules in bug #517886.
https://bugzilla.redhat.com/503156
qemu VNC :: xterm inside VM shows garbled text
https://bugzilla.redhat.com/501131
qemu segfault when VNC client disconnects
Both of these VNC problems have been fixed upstream, but not yet
on the stable-0.10 branch.
https://bugzilla.redhat.com/514241
Evaluate the need for qemu's virtio_net TX mitigation timer
In RHEL5, after a whole pile of benchmarking and procrastination,
we disabled the TX mitigation timer. However, the situation with
recent host kernels is very different, so we need to look into it
again for Fedora 12 and upstream.
https://bugzilla.redhat.com/515024
KVM USB passthrough - device reset messages in host dmesg
It looks likes something screwy is causing assigned USB devices to
be reset over and over by the host.
https://bugzilla.redhat.com/517884
USB hard disks can't be specified using qemu's -drive option
Dan Berrange points out that because USB drivers have their own
option, the usual drive options cannot be specified.
https://bugzilla.redhat.com/518032
Restoring a qemu guest from a saved state file using -incoming
sometimes fails and hangs
With libvirt-tck, a qemu guest hangs while restoring a saved state
file. Not confirmed yet whether this is TCG specific.
== libvirt ==
https://bugzilla.redhat.com/518102
libvirt name/uuid uniqueness checks are broken
Some issues with name/uuid uniqueness checking uncovered by
libvirt-tck.
https://bugzilla.redhat.com/465532
RFE: libvirt should support KVM huge page backed memory
This is a bugzilla for tracking part of the KVM Huge Page Backed
Memory feature.
https://bugzilla.redhat.com/496537
RFE: Support virDomainReboot() for qemu/kvm guests
https://bugzilla.redhat.com/503184
Add system_reboot to qemu
There's been some discussion on qemu-devel about how libvirt could
implement virDomainReboot() - the latest conclusion seems to be
that it should do system_powedown, poll info status and then do
system_reset.
https://bugzilla.redhat.com/517230
Guest VM freeze during live migration
A Fedora 11 live migration failure using libvirt. Needs someone to
debug it.
== virt-manager ==
https://bugzilla.redhat.com/499703
virt-manager should run stats refresh operation in a background
thread per connection
https://bugzilla.redhat.com/502204
virt-manager's dialog to connect an existing CD-ROM to an ISO does
not use storage pool interface
https://bugzilla.redhat.com/503784
memory/vcpus changes in virt-manager do not persist across
libvirtd restart
https://bugzilla.redhat.com/476956
RFE: ability to add serial device
Some of the bugs fixed by the virt-manager-0.8.0 release.
Resolved Bugs
=============
== misc ==
https://bugzilla.redhat.com/514023
dracut: support booting from KVM virtio devices
dracut needed a hack to pull in virtio_pci, otherwise the initrds
it produced wouldn't work for KVM guests.
https://bugzilla.redhat.com/512206
Disable net.bridge.bridge-nf-call-*tables by default
Finally we have netfilter on the bridge disabled by default in
F-12.
== kernel ==
https://bugzilla.redhat.com/514901
kvm virtio_blk errors - "end_request: I/O error, dev vda, sector 0"
This issue turned out to be that device-mapper is submitting empty
barrier requests in 2.6.31 and the block layer is passing them
through to virtio-blk, even though virtio-blk doesn't support
barriers. Fix sent upstream and applied in rawhide.
https://bugzilla.redhat.com/505695
Poor KVM guest performance doing kernel builds (100+% overhead,
w/ 8vcpu and virtio)
This issue was resolved by using rotational mode in the guest,
deadline scheduler in the host and -drive cache=none.
https://bugzilla.redhat.com/499352
Re-enable CONFIG_DMAR_DEFAULT_ON
dwmw2 has applied some VT-d fixes and workarounds to the F-12
kernel and enabled it by default again. No need for intel_iommu=on
any more.
https://bugzilla.redhat.com/510304
kernel oops/panic: IP: [<c048a9f8>] __bounce_end_io_read+0x88/0xf8
This F10 guest oops was fixed by backporting a virtio-blk patch to
disable bouncing highmem requests.
== qemu ==
https://bugzilla.redhat.com/509772
'qemu-img convert' failed to convert an image which contains a
backing file
Akkarit Sangpetch found this bug with qemu in virt-preview, came
up with a patch, sent it upstream and the fix was included in
qemu-0.11.0-rc1. That's how it should be done! :-)
https://bugzilla.redhat.com/516022
virtio-net fails to transmit any packets, gives "Network is
unreachable" errors
This F-12 virtio_net failure was only reproducible using
libguestfs, but after some bisection it was narrowed down to a
problem with qemu-kvm's GSO support. Fix sent upstream and applied
in rawhide.
https://bugzilla.redhat.com/514899
Unable to boot using virtio disk
Rawhide qemu-kvm briefly had a broken extboot image which caused
booting from virtio disks to fail.
https://bugzilla.redhat.com/516543
qemu-kvm segfaults when run inside another virtual machine
Rich Jones has found yet another TCG bug by running libguestfs
'make check' inside Koji. Rich bisected the problem, posted a fix
upstream and applied the fix in rawhide.
https://bugzilla.redhat.com/517866
Allow kvm modules to be blacklisted via modprobe.conf
Lubomir Rintel fixed kvm.modules to use 'modprobe -b' so that kvm
modules can be blacklisted via modprobe.conf.
https://bugzilla.redhat.com/517571
[QEMU] file /etc/udev/rules.d/80-kvm.rules* is set to executable
Joachim Namislow noted that the permissions on 80-kvm.rules were
incorrect in rawhide.
== libvirt ==
https://bugzilla.redhat.com/499970
RFE: port libvirt to PolicyKit 1.0
PolicyKit has changed its ABI and wants all apps to port to the
new ABI in Fedora 12. Dan Berrange has come up with a patch for
libvirt and added it to rawhide.
https://bugzilla.redhat.com/489481
Useless "domain didn't show up" error when starting a guest with
too much RAM
Fixed in 0.6.4. Not attempting to backport to F11.
https://bugzilla.redhat.com/509458
allow libvirt.so to be installed without libvirtd
The libvirt-client sub-package has now been split out from the
main libvirt package.
https://bugzilla.redhat.com/506590
libvirt should ignore NUMA cells with missing topology
It seems the numactl fix wasn't enough here for F-11 users, so
danpb backported the libvirt fix.
https://bugzilla.redhat.com/516497
no virbr0 with libvirt-0.7.0-2
On machines where ipv6 disabled, latest libvirt was failing to
start any virtual networks. Fixed now in rawhide.
https://bugzilla.redhat.com/499669
libvirt QEMU driver is using old pci_add/pci_del syntax
Fedora 11 libvirt now supports the newer qemu hotplug syntax
thanks to danpb.
https://bugzilla.redhat.com/516187
libvirt should run qemu 'cont' command on successful migration
finish
Chris Lalancette noticed that newer qemu needs a "cont" command to
be issued when the migration has finished. This fix has now been
backported to F-11 and F-12.
https://bugzilla.redhat.com/507405
virsh: renaming of guests creates a copy
danpb backported a fix to F-11 which disallows re-naming guests.
https://bugzilla.redhat.com/518091
libvirt virEnumFromString crashes on F11 with Xen 3.4.x when
starting virt-viewer
A libvirt segfault with latest Xen. The libvirt-0.6.2-17.fc11
updates fixes this.
== python-virtinst ==
https://bugzilla.redhat.com/505317
virtinst: make SLES11 guests use virtio by default
Fixed in rawhide now by 0.500.0, still might be worth backporting
to F-11.
https://bugzilla.redhat.com/511071
RFE: default to qcow2 rather than "raw" for virtual disk file
Now that qcow2 performance is much improved, perhaps we should
consider switching to it by default in Fedora 13.
https://bugzilla.redhat.com/517151
virtinst creates cdrom device using virtio rather than IDE
When creating a guest, virtinst is now erroneously trying to
create a virtio cdrom rather than an IDE cdrom.
== virt-manager ==
https://bugzilla.redhat.com/517548
virt-manager migration failure - destination URI, not hostname,
should be passed to vm.prepare()
Migration using virt-manager appears to be totally broken because
of a hostname/URI mixup.
https://bugzilla.redhat.com/516116
virt-manager error caused by connect_cdrom() : unsupported driver
name 'file'
Looks like connecting a cdrom to a kvm guest in virt-manager is
broken; we're generating invalid XML for the libvirt qemu driver.
https://bugzilla.redhat.com/517289
[PATCH] Fix virt-manager addhardware.py hostdev error handling
Paul Frields found and fixed a bug in virt-manager USB device
assignment error handling.
https://bugzilla.redhat.com/517293
virt-manager storage browser ISO/disk callback mixup
Tim Waugh found this nice bug in the latest virt-manager.
https://bugzilla.redhat.com/513494
RFE: add a virt-manager first-time wizard for installing kvm/xen
Mairin Duffy suggests that virt-manager should have a wizard to
allow people to install kvm/xen when they first run it.
https://bugzilla.redhat.com/517664
virt-manager ignores "Host does not support any virtualization
options" error
A related issue is that the "Add VM" wizard currently just has
greyed out buttons if no kvm/xen is installed. An error in
virt-manager.log is the only way the user can figure out what's
wrong.
https://bugzilla.redhat.com/517778
virt-manager hangs waiting for VNC ssh tunnel to exit
For at least one user, virt-manager hangs when you close a guest
console as it waits for an SSH process to exit.
https://bugzilla.redhat.com/513949
virt-manager scaling should maintain the aspect ratio of the
display
virt-manager needs to copy some of the scaling improvements
recently made in virt-viewer.
More information about the Fedora-virt
mailing list