[Fedora-xen] Problem found with workaround, was Re: Booting rootfs instead of disk image, question on xen modules

[Paul Wouters]

On Fri, 10 Nov 2006, Paul Wouters wrote:

[ ExecSum since it's a long post: xenU created from xenguest-isntall.py
  has different initrd from dom0 though with the same version/build/rpmsig,
  and rpm has a nasty bug]

Following up on myself, since i now know what happened:

> Loading ohci-hcd.ko module
> Loading ehci-hcd.ko module
> Loading jbd.ko module
> Loading ext3.ko module
> Loading scsi_mod.ko module
> SCSI subsystem initialized

Note the missing "Loading xenblk" which was visible in the boot of the guest
that was started with pygrub using a disk image created on the dom0 with
xenguest-install.py.

I decided to find out where it got loaded from. It couldn't be from
the filesystem, so it had to be initrd. But both were using the same
initrd? Well sort of. pygrub extracted one from the diskimage, and
I grabbed it from /boot on the dom0. What I found were two different
initrd files with the SAME build number. I copied it from xenU to dom0:

[root at echelon boot]# ls -l initrd-2.6.18-1.2798.fc6*xen*
-rw------- 1 root root 2177639 Nov 10 00:00 initrd-2.6.18-1.2798.fc6xen.img-from-xenguest-install
-rw------- 1 root root 1522165 Oct 28 00:34 initrd-2.6.18-1.2798.fc6xen.img-from-dom0

The original date of the xenguest=install version before i copied it was: "Oct 28 16:37"

[root at echelon boot]# md5sum initrd-2.6.18-1.2798.fc6*xen*
8b06aa9e364bd1ac5379e652095a4d3e  initrd-2.6.18-1.2798.fc6xen.img-from-xenguest-install
6b591a095b5d4e55641d30c25702e272  initrd-2.6.18-1.2798.fc6xen.img-from-dom0

They both claim to come from the package kernel-xen-2.6.18-1.2798.fc6

Both the diskimage based xen's rpm -V, and the rpm -V on the dom0
claimed the package was not altered?

Checking the rpms, I find both as:

# rpm -qi kernel-xen-2.6.18-1.2798.fc6
Name        : kernel-xen                   Relocations: (not relocatable)
Version     : 2.6.18                            Vendor: Red Hat, Inc.
Release     : 1.2798.fc6                    Build Date: Mon 16 Oct 2006 03:57:26 PM EDT
Install Date: Sat 28 Oct 2006 04:37:14 PM EDT      Build Host: hs20-bc2-4.build.redhat.com
Group       : System Environment/Kernel     Source RPM: kernel-2.6.18-1.2798.fc6.src.rpm
Size        : 46862330                         License: GPLv2

So how can they both verify to the same version, with rpm -V not complaining,
while containing different initrd images, yet having the same build date?

Just to show what I did, so that I won't go insane:

[root at echelon boot]# rpm -qf /boot/initrd-2.6.18-1.2798.fc6xen.img
kernel-xen-2.6.18-1.2798.fc6
[root at echelon boot]# rpm -qa|grep kernel-xen
kernel-xen-devel-2.6.18-1.2798.fc6
kernel-xen-devel-2.6.18-1.2741.fc6
kernel-xen-2.6.18-1.2741.fc6
kernel-xen-2.6.18-1.2798.fc6
[root at echelon boot]# ls -l /boot/initrd-2.6.18-1.2798.fc6*
-rw------- 1 root root 1521809 Oct 28 00:36 /boot/initrd-2.6.18-1.2798.fc6.img
-rw------- 1 root root 2177639 Nov 10 00:30 /boot/initrd-2.6.18-1.2798.fc6xen.img
-rw------- 1 root root 1522165 Oct 28 00:34 /boot/initrd-2.6.18-1.2798.fc6xen.img-broken
-rw------- 1 root root 2177639 Nov 10 00:28 /boot/initrd-2.6.18-1.2798.fc6xen.img-from-xenguest
[root at echelon boot]# rpm -V kernel-xen-2.6.18-1.2798.fc6
[root at echelon boot]# cp /boot/initrd-2.6.18-1.2798.fc6xen.img-broken /boot/initrd-2.6.18-1.2798.fc6xen.img
[root at echelon boot]# ls -l /boot/initrd-2.6.18-1.2798.fc6*
-rw------- 1 root root 1521809 Oct 28 00:36 /boot/initrd-2.6.18-1.2798.fc6.img
-rw------- 1 root root 1522165 Nov 10 00:35 /boot/initrd-2.6.18-1.2798.fc6xen.img
-rw------- 1 root root 1522165 Oct 28 00:34 /boot/initrd-2.6.18-1.2798.fc6xen.img-broken
-rw------- 1 root root 2177639 Nov 10 00:28 /boot/initrd-2.6.18-1.2798.fc6xen.img-from-xenguest
[root at echelon boot]# rpm -V kernel-xen-2.6.18-1.2798.fc6
[root at echelon boot]#

So either rpm is broken in -qf mode, or it is broken in -V mode...

[root at echelon boot]# rpm -qf /boot/initrd-2.6.18-1.2798.fc6xen.img
kernel-xen-2.6.18-1.2798.fc6
[root at echelon boot]# rpm -ql kernel-xen-2.6.18-1.2798.fc6 |grep initrd
[root at echelon boot]#

And as a test:
[root at echelon boot]# rpm -qf /boot/initrd-2.6.18-1.2798.fc6xen.img-broken
file /boot/initrd-2.6.18-1.2798.fc6xen.img-broken is not owned by any package

I guess it is rpm -qf that is broken? Or some advanced "should never be visible"
magic is in use.

Replacing the dom0 version with the xenguest installed version fixed my
problem. The rootfs based xen now boots fine.

And this leads me to a final question. Why are the xen modules loaded
through initrd to begin with? kernel size can't be an issue, since all
hardware drivers are gone from the xenU - only the xen virtual drivers
are needed. Can't we just build that stuff inline, and save ourselves
a lot of trouble booting. It might not be the elegant CS theoretical
perfect solution, but it works in many more practical situations, like
this one with a voodoo initrd-2.6.18.-1.2798.fc6xen.img appearing.

Paul