[fedora-virt] Fedora virt status
markmc at redhat.com
Fri Aug 21 11:55:08 UTC 2009
It's been a busy seven weeks or so since I sent the last one of
these. I'll try not to leave such a big gap between status reports in
The Fedora 12 Alpha release is now baked and will be released next
week on August 25th.
The next big deadline coming up is the Final Development freeze on
September 29th. After that date, only important bug fixes will be
The final list of virt features for Fedora 12 looks like:
* KVM Huge Page Backed Memory
* KVM NIC Hotplug
* KVM qcow2 Performance
* KVM Stable Guest ABI
* Network Interface Management
(Note, FESCo didn't approve TCK as a feature, but that should't stop
us pimping it :-)
F-12 Changes to System Defaults
There are a couple of changes to Fedora 12 system defaults that are
related to virtualization:
For security and performance reasons, iptables rules are no longer
applied by default to frames forwarded across linux kernel ethernet
bridges. See bug #512206 for more details on the rationale behind
Historically, uids and gids 0-100 are reserved for specific system
accounts and allocated via the uidgid file in the setup
package. This space has now been exhausted and 0-200 is now
reserved. This should not be an issue on most systems because
dynamically allocated system accounts are usually allocated
downwards from 499. See bug 515779 and bug #511957 for more details.
We already have quite a number of features planned for Fedora 13. See:
One of the most interesting of those is Michael Tsirkin's "kernel
acceleration for KVM networking":
The idea is to add a kernel module which much more efficiently takes
care of the packet handling part of the virtio_net host backend. The
progress of this feature can be followed on the Linux Foundation
virtualization mailing list:
Several new releases of various virt bits have been released recently:
Fedora Weekly News
Unlike me, Dale Bewley is no slacker and has kept the FWN updates
Rich Jones announced that libguestfs has its own (very busy) mailing
Also of note is that the virt-df utility has now been re-written to
The badly named et-mgmt-tools mailing list has been deprecated in
favour of a new virt-tools-list:
This list originally came into being as a place for discussing
projects under Red Hat's 'emerging technology' moniker, hence the
prefix 'et-'. In retrospect this was a really bad choice of names
for a mailing list and causes endless confusion for people wrt what
to discuss where. Most of the emerging technology projects have
lists of their own (cobbler, augeas, libguestfs, libvirt) and it is
about time that virt-manager and friends joined them.
To that end we have created a new mailing list
'virt-tools-list'. This will be the new home for all developer &
user discussions relating to the following applications
M A Young, Pasi Kärkkäinen and others are continuing to work hard
testing builds of latest upstream pv_ops Dom0. See the fedora-xen
mailing list archives:
Gerd Hoffman has updated Fedora 12 to xen-3.4.1:
Fedora 12's Xen DomU support has seen a number of problems.
Switch bzImage from LZMA back to gzip compression so Xen can load
Fedora kernels again
It turns out that Fedora switched their bzImage format from
gzip to LZMA, which the Xen loader doesn't support. This has been
reverted until Fedora 13, giving Xen a chance to catch up.
Chris Lalancette quickly took on the task of making sure that we have
LZMA support in the Xen domain builder. Patches for this are upstream
now and we just need them pulled into Fedora 12:
Add xen domain builder support for bzImage lzma/bzip2 compression
However, we're not done yet. The F12 kernel still doesn't boot as a
2.6.31-rc1 xen domU crashes early during boot
It now turns out that the F12 kernel crashes during boot in Xen
DomU. Jeremy Fitzhardinge has come up with patches to fix at least
some of this, but it sounds like there are more dragons lurking
Michael Schmidt points out this xenfb issue:
So it crashes during Xen framebuffer initialization. And indeed,
disabling CONFIG_XEN_FBDEV_FRONTEND helps, the kernel then boots
Fedora QEMU/KVM Security
There are several things to bear in mind wrt to libvirt's support for
qemu/kvm and security:
1) The qemu process now runs as the qemu user, not root. This
reduces the ability of the process to attack the host if it is
compromised. However, users should be aware of the potential for
issues with e.g. directories having the wrong permissions.
2) qemu processes are also confined using SELinux sVirt
protection. This reduces the ability of the process to attack
other qemu processes if it is compromised. Again, though, there
is the potential for users to see problems caused by e.g. files
not being labelled correctly.
Dan Berrange prepared a comprehensive set of docs on the security
architecture for libvirt's qemu driver:
Some of the recently active bug reports in this are include:
'groupadd -r' allocates gids upwards
login.defs/SYS_UID_MIN should be 200
The qemu uidgid reservation is 107, but 'useradd/groupadd -r' are
still allocating out of the 100-500 range. It wasn't such a big
problem when they used to allocate downwards from the top of the
system accounts range, but this behaviour changed recently.
Make the /dev/kvm device world accessible to all users by default
Create a kvm user account and kvm group
QEMU driver should run all QEMU VMs as non-root system account
All done by danpb for F-12 as part of the VirtPrivileges
Directory permissions on volume group directory too restrictive
The VirtPrivileges feature requires that the LVM volume group
directory permissions are relaxed a bit. Apparently this should be
magically fixed by lvm using udev but, although it has switched to
udev now, it doesn't seem to have changed anything.
libvirt fails to start guest - Failed to set security label
An selinux-policy regression in Fedora 12 caused libvirt to
break. Fixed in rawhide now.
SELinux is preventing qemu-kvm (svirt_t) "setrlimit" svirt_t
An SELinux setrlimit() denial is causing qemu to fail to start for
some F-11 users. At first, we had no idea where setrlimit() is
being called from but Jerry James figured out that it was glibc.
It turns out that glibc has a workaround for the fact that
/dev/pts was incorrectly mounted in F-11 and an selinux-policy
update to allow glibc to run that workaround has now been pushed.
It also turns out that qemu isn't setting some file descriptors to
CLOEXEC and this is causing selinux problems when pt_chown is
Allow svirt images to create sock_file in svirt_var_run_t
A Fedora 11 selinux-policy update needed to use the virt-preview
version of libvirt.
libvirt only relabels disks *after* hotplugging them into QEMU
A fix for this issue has been backported to F-11. It fixes
problems like not being able to attach a dvd/cdrom to a guest in
libvirt cannot re-label a disk image under an NTFS partition
Because NTFS doesn't support xattrs, svirt cannot start a guest
with disk images on an NTFS partition.
libvirt is not chowning kernel/initrd images before launching qemu
As part of the F-12 VirtPrivileges feature we started running the
qemu process unprivileged, but we neglected to chown kernel and
initrd images before launching qemu. Fixed now in F-12 Alpha.
libvirt fails to start guest on NFS even when sebool virt_use_nfs
David Lutterkort notes that libvirt is defeating the purpose of
the virt_use_nfs sebool by refusing to start a guest if it can't
relabel its disk images.
libvirt needs to better handle chown-ing images on NFS shares
Now that we're chown-ing images before starting guests, we need to
make various improvements in order to handle NFS shares.
libvirt/netcf loads modprobe.conf and others - AVC messages
(preventing libvirtd (virtd_t) "getattr" modules_conf_t)
libvirt's new network interface configuration support
(unsupringly) touches a bunch of files in /etc, so we need policy
updates to allow libvirtd to do that.
libvirt fails to start guest with qemu configured to run as
There seems to be a selinux-policy issue where if libvirt is
configured to run guests as root/root, they fail to transition to
svirt_t. Strangely, the AVCs persist when you change the
configuration back until you reboot, even though the transitions
do appear to be succeeding.
Aside from the AVCs, we need to make libvirt chown various
directories to the user is going to run qemu as.
virt-manager should warn if guest images will are not readable by
If a user downloads an ISO to her homedir and tries to start a
guest using it, it fails because qemu doesn't have permissions to
the homedir. We could warn the user of this common scenario.
KVM PCI Device Assignment
A number of improvements to the feature introduced in Fedora 11 are
now available as an update:
libvirt should allow PCI PM reset on multi-function devices
libvirt does not automatically re-attach an assigned device in the
host after guest shutdown
libvirt should be able to reset a PCI function even if it causes
other unused devices/functions to be reset
libvirt should allow PCI PM reset on multi-function devices
Also, tieing in with the recent work to add KVM NIC hotplug support to
libvirt, we now have support in Fedora 12 for assigned device hotplug:
Add support to libvirt for KVM PCI device assignment hotplug
The last while has seen a huge churn of bugs in bugzilla, leaving us
with a DOOM-O-METER of 217 now. Seven weeks ago we were up to 250.
If you're looking to help getting this number down even further, the
place to start is the Fedora 12 blocker and target lists:
== misc ==
Implement support for CLONE_IO
Request for glibc to support CLONE_IO. Uli suggests that CLONE_IO
should be used by default. Avi suggests that it shouldn't.
== kernel ==
rotational mode is much faster for virtio-blk disks, but uses
non-rotational mode by default
This issue is still ongoing, we need to get the default changed.
Unable to boot using qemu-kvm and gPXE from virt-preview
We need a backport of a kvm.ko fix in order to be able to use gPXE
on an F-11 host.
2.6.30 kernel stopped supporting xattrs on hugetlbfs
This issue is preventing libvirt from using SELinux labels to
enforce separation between qemu guests using huge page backed
memory. John Cooper is working to fix this for the KVM Huge Page
Backed Memory feature in Fedora 12.
KSM breaks encryption 157 > kernel > 139 - KSM support now
A recent set of KSM changes from upstream has caused a regression
with encrypted volumes. KSM has been disabled until this is
2.6.31 virtio_net oops in skb_copy_from_linear_data_offset()
James Laska hit this nice oops during an F12 guest install over
== qemu ==
Enable qemu sound devices to tunnel over VNC
Allow sounds devices to be used with svirt - tunnel sound over VNC
These bugs have been moved to F13VirtTarget now that the feature
has been punted to Fedora 13.
Guest clock is running aprox. 3 seconds before host clock
Strange problem with the guest clock consistently being a few
seconds behind the host clock. Removing hwclock from the system
reduces the offset to below one second. This is beginning to look
like a fundamental problem with the rtc resolution and using
hwclock to sync the system time during boot. Glauber proposes
removing 88-clock.rules in bug #517886.
qemu VNC :: xterm inside VM shows garbled text
qemu segfault when VNC client disconnects
Both of these VNC problems have been fixed upstream, but not yet
on the stable-0.10 branch.
Evaluate the need for qemu's virtio_net TX mitigation timer
In RHEL5, after a whole pile of benchmarking and procrastination,
we disabled the TX mitigation timer. However, the situation with
recent host kernels is very different, so we need to look into it
again for Fedora 12 and upstream.
KVM USB passthrough - device reset messages in host dmesg
It looks likes something screwy is causing assigned USB devices to
be reset over and over by the host.
USB hard disks can't be specified using qemu's -drive option
Dan Berrange points out that because USB drivers have their own
option, the usual drive options cannot be specified.
Restoring a qemu guest from a saved state file using -incoming
sometimes fails and hangs
With libvirt-tck, a qemu guest hangs while restoring a saved state
file. Not confirmed yet whether this is TCG specific.
== libvirt ==
libvirt name/uuid uniqueness checks are broken
Some issues with name/uuid uniqueness checking uncovered by
RFE: libvirt should support KVM huge page backed memory
This is a bugzilla for tracking part of the KVM Huge Page Backed
RFE: Support virDomainReboot() for qemu/kvm guests
Add system_reboot to qemu
There's been some discussion on qemu-devel about how libvirt could
implement virDomainReboot() - the latest conclusion seems to be
that it should do system_powedown, poll info status and then do
Guest VM freeze during live migration
A Fedora 11 live migration failure using libvirt. Needs someone to
== virt-manager ==
virt-manager should run stats refresh operation in a background
thread per connection
virt-manager's dialog to connect an existing CD-ROM to an ISO does
not use storage pool interface
memory/vcpus changes in virt-manager do not persist across
RFE: ability to add serial device
Some of the bugs fixed by the virt-manager-0.8.0 release.
== misc ==
dracut: support booting from KVM virtio devices
dracut needed a hack to pull in virtio_pci, otherwise the initrds
it produced wouldn't work for KVM guests.
Disable net.bridge.bridge-nf-call-*tables by default
Finally we have netfilter on the bridge disabled by default in
== kernel ==
kvm virtio_blk errors - "end_request: I/O error, dev vda, sector 0"
This issue turned out to be that device-mapper is submitting empty
barrier requests in 2.6.31 and the block layer is passing them
through to virtio-blk, even though virtio-blk doesn't support
barriers. Fix sent upstream and applied in rawhide.
Poor KVM guest performance doing kernel builds (100+% overhead,
w/ 8vcpu and virtio)
This issue was resolved by using rotational mode in the guest,
deadline scheduler in the host and -drive cache=none.
dwmw2 has applied some VT-d fixes and workarounds to the F-12
kernel and enabled it by default again. No need for intel_iommu=on
kernel oops/panic: IP: [<c048a9f8>] __bounce_end_io_read+0x88/0xf8
This F10 guest oops was fixed by backporting a virtio-blk patch to
disable bouncing highmem requests.
== qemu ==
'qemu-img convert' failed to convert an image which contains a
Akkarit Sangpetch found this bug with qemu in virt-preview, came
up with a patch, sent it upstream and the fix was included in
qemu-0.11.0-rc1. That's how it should be done! :-)
virtio-net fails to transmit any packets, gives "Network is
This F-12 virtio_net failure was only reproducible using
libguestfs, but after some bisection it was narrowed down to a
problem with qemu-kvm's GSO support. Fix sent upstream and applied
Unable to boot using virtio disk
Rawhide qemu-kvm briefly had a broken extboot image which caused
booting from virtio disks to fail.
qemu-kvm segfaults when run inside another virtual machine
Rich Jones has found yet another TCG bug by running libguestfs
'make check' inside Koji. Rich bisected the problem, posted a fix
upstream and applied the fix in rawhide.
Allow kvm modules to be blacklisted via modprobe.conf
Lubomir Rintel fixed kvm.modules to use 'modprobe -b' so that kvm
modules can be blacklisted via modprobe.conf.
[QEMU] file /etc/udev/rules.d/80-kvm.rules* is set to executable
Joachim Namislow noted that the permissions on 80-kvm.rules were
incorrect in rawhide.
== libvirt ==
RFE: port libvirt to PolicyKit 1.0
PolicyKit has changed its ABI and wants all apps to port to the
new ABI in Fedora 12. Dan Berrange has come up with a patch for
libvirt and added it to rawhide.
Useless "domain didn't show up" error when starting a guest with
too much RAM
Fixed in 0.6.4. Not attempting to backport to F11.
allow libvirt.so to be installed without libvirtd
The libvirt-client sub-package has now been split out from the
main libvirt package.
libvirt should ignore NUMA cells with missing topology
It seems the numactl fix wasn't enough here for F-11 users, so
danpb backported the libvirt fix.
no virbr0 with libvirt-0.7.0-2
On machines where ipv6 disabled, latest libvirt was failing to
start any virtual networks. Fixed now in rawhide.
libvirt QEMU driver is using old pci_add/pci_del syntax
Fedora 11 libvirt now supports the newer qemu hotplug syntax
thanks to danpb.
libvirt should run qemu 'cont' command on successful migration
Chris Lalancette noticed that newer qemu needs a "cont" command to
be issued when the migration has finished. This fix has now been
backported to F-11 and F-12.
virsh: renaming of guests creates a copy
danpb backported a fix to F-11 which disallows re-naming guests.
libvirt virEnumFromString crashes on F11 with Xen 3.4.x when
A libvirt segfault with latest Xen. The libvirt-0.6.2-17.fc11
updates fixes this.
== python-virtinst ==
virtinst: make SLES11 guests use virtio by default
Fixed in rawhide now by 0.500.0, still might be worth backporting
RFE: default to qcow2 rather than "raw" for virtual disk file
Now that qcow2 performance is much improved, perhaps we should
consider switching to it by default in Fedora 13.
virtinst creates cdrom device using virtio rather than IDE
When creating a guest, virtinst is now erroneously trying to
create a virtio cdrom rather than an IDE cdrom.
== virt-manager ==
virt-manager migration failure - destination URI, not hostname,
should be passed to vm.prepare()
Migration using virt-manager appears to be totally broken because
of a hostname/URI mixup.
virt-manager error caused by connect_cdrom() : unsupported driver
Looks like connecting a cdrom to a kvm guest in virt-manager is
broken; we're generating invalid XML for the libvirt qemu driver.
[PATCH] Fix virt-manager addhardware.py hostdev error handling
Paul Frields found and fixed a bug in virt-manager USB device
assignment error handling.
virt-manager storage browser ISO/disk callback mixup
Tim Waugh found this nice bug in the latest virt-manager.
RFE: add a virt-manager first-time wizard for installing kvm/xen
Mairin Duffy suggests that virt-manager should have a wizard to
allow people to install kvm/xen when they first run it.
virt-manager ignores "Host does not support any virtualization
A related issue is that the "Add VM" wizard currently just has
greyed out buttons if no kvm/xen is installed. An error in
virt-manager.log is the only way the user can figure out what's
virt-manager hangs waiting for VNC ssh tunnel to exit
For at least one user, virt-manager hangs when you close a guest
console as it waits for an SSH process to exit.
virt-manager scaling should maintain the aspect ratio of the
virt-manager needs to copy some of the scaling improvements
recently made in virt-viewer.
More information about the Fedora-virt