[Fedora-xen] Problem found with workaround, was Re: Booting rootfs instead of disk image, question on xen modules

Fri Nov 10 13:50:16 UTC 2006

Paul Wouters wrote:
> On Fri, 10 Nov 2006, Paul Wouters wrote:
> 
> [ ExecSum since it's a long post: xenU created from xenguest-isntall.py
>   has different initrd from dom0 though with the same version/build/rpmsig,
>   and rpm has a nasty bug]
> 
> Following up on myself, since i now know what happened:
> 
>> Loading ohci-hcd.ko module
>> Loading ehci-hcd.ko module
>> Loading jbd.ko module
>> Loading ext3.ko module
>> Loading scsi_mod.ko module
>> SCSI subsystem initialized
> 
> Note the missing "Loading xenblk" which was visible in the boot of the guest
> that was started with pygrub using a disk image created on the dom0 with
> xenguest-install.py.
> 
> I decided to find out where it got loaded from. It couldn't be from
> the filesystem, so it had to be initrd. But both were using the same
> initrd? Well sort of. pygrub extracted one from the diskimage, and
> I grabbed it from /boot on the dom0. What I found were two different
> initrd files with the SAME build number. I copied it from xenU to dom0:
> 
> [root at echelon boot]# ls -l initrd-2.6.18-1.2798.fc6*xen*
> -rw------- 1 root root 2177639 Nov 10 00:00 initrd-2.6.18-1.2798.fc6xen.img-from-xenguest-install
> -rw------- 1 root root 1522165 Oct 28 00:34 initrd-2.6.18-1.2798.fc6xen.img-from-dom0
> 
> The original date of the xenguest=install version before i copied it was: "Oct 28 16:37"
> 
> [root at echelon boot]# md5sum initrd-2.6.18-1.2798.fc6*xen*
> 8b06aa9e364bd1ac5379e652095a4d3e  initrd-2.6.18-1.2798.fc6xen.img-from-xenguest-install
> 6b591a095b5d4e55641d30c25702e272  initrd-2.6.18-1.2798.fc6xen.img-from-dom0
> 
> They both claim to come from the package kernel-xen-2.6.18-1.2798.fc6
> 
> Both the diskimage based xen's rpm -V, and the rpm -V on the dom0
> claimed the package was not altered?
> 
> Checking the rpms, I find both as:
> 
> # rpm -qi kernel-xen-2.6.18-1.2798.fc6
> Name        : kernel-xen                   Relocations: (not relocatable)
> Version     : 2.6.18                            Vendor: Red Hat, Inc.
> Release     : 1.2798.fc6                    Build Date: Mon 16 Oct 2006 03:57:26 PM EDT
> Install Date: Sat 28 Oct 2006 04:37:14 PM EDT      Build Host: hs20-bc2-4.build.redhat.com
> Group       : System Environment/Kernel     Source RPM: kernel-2.6.18-1.2798.fc6.src.rpm
> Size        : 46862330                         License: GPLv2
> 
> So how can they both verify to the same version, with rpm -V not complaining,
> while containing different initrd images, yet having the same build date?
> 
> Just to show what I did, so that I won't go insane:
> 
> [root at echelon boot]# rpm -qf /boot/initrd-2.6.18-1.2798.fc6xen.img
> kernel-xen-2.6.18-1.2798.fc6
> [root at echelon boot]# rpm -qa|grep kernel-xen
> kernel-xen-devel-2.6.18-1.2798.fc6
> kernel-xen-devel-2.6.18-1.2741.fc6
> kernel-xen-2.6.18-1.2741.fc6
> kernel-xen-2.6.18-1.2798.fc6
> [root at echelon boot]# ls -l /boot/initrd-2.6.18-1.2798.fc6*
> -rw------- 1 root root 1521809 Oct 28 00:36 /boot/initrd-2.6.18-1.2798.fc6.img
> -rw------- 1 root root 2177639 Nov 10 00:30 /boot/initrd-2.6.18-1.2798.fc6xen.img
> -rw------- 1 root root 1522165 Oct 28 00:34 /boot/initrd-2.6.18-1.2798.fc6xen.img-broken
> -rw------- 1 root root 2177639 Nov 10 00:28 /boot/initrd-2.6.18-1.2798.fc6xen.img-from-xenguest
> [root at echelon boot]# rpm -V kernel-xen-2.6.18-1.2798.fc6
> [root at echelon boot]# cp /boot/initrd-2.6.18-1.2798.fc6xen.img-broken /boot/initrd-2.6.18-1.2798.fc6xen.img
> [root at echelon boot]# ls -l /boot/initrd-2.6.18-1.2798.fc6*
> -rw------- 1 root root 1521809 Oct 28 00:36 /boot/initrd-2.6.18-1.2798.fc6.img
> -rw------- 1 root root 1522165 Nov 10 00:35 /boot/initrd-2.6.18-1.2798.fc6xen.img
> -rw------- 1 root root 1522165 Oct 28 00:34 /boot/initrd-2.6.18-1.2798.fc6xen.img-broken
> -rw------- 1 root root 2177639 Nov 10 00:28 /boot/initrd-2.6.18-1.2798.fc6xen.img-from-xenguest
> [root at echelon boot]# rpm -V kernel-xen-2.6.18-1.2798.fc6
> [root at echelon boot]#
> 
> So either rpm is broken in -qf mode, or it is broken in -V mode...
> 
> [root at echelon boot]# rpm -qf /boot/initrd-2.6.18-1.2798.fc6xen.img
> kernel-xen-2.6.18-1.2798.fc6
> [root at echelon boot]# rpm -ql kernel-xen-2.6.18-1.2798.fc6 |grep initrd
> [root at echelon boot]#
> 
> And as a test:
> [root at echelon boot]# rpm -qf /boot/initrd-2.6.18-1.2798.fc6xen.img-broken
> file /boot/initrd-2.6.18-1.2798.fc6xen.img-broken is not owned by any package
> 
> I guess it is rpm -qf that is broken? Or some advanced "should never be visible"
> magic is in use.
> 
> Replacing the dom0 version with the xenguest installed version fixed my
> problem. The rootfs based xen now boots fine.
> 
> And this leads me to a final question. Why are the xen modules loaded
> through initrd to begin with? kernel size can't be an issue, since all
> hardware drivers are gone from the xenU - only the xen virtual drivers
> are needed. Can't we just build that stuff inline, and save ourselves
> a lot of trouble booting. It might not be the elegant CS theoretical
> perfect solution, but it works in many more practical situations, like
> this one with a voodoo initrd-2.6.18.-1.2798.fc6xen.img appearing.
> 
> Paul
> 
> --
> Fedora-xen mailing list
> Fedora-xen at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-xen

Paul,
     The reason that you are seeing two different initrd's is that the initrd is actually packaged as a file in the RPM.  Since everyone's hardware is different, including every single driver in an initrd would be a little silly.  After the kernel is finished installing, it runs "mkinitrd", looking at your hardware (specifically, /etc/modprobe.conf) and generating an initrd for your particular hardware.  In this case, because mkinitrd was run on the dom0, it crucially included the modules needed to run dom0, but not the ones necessary for domU (like, as you said, xenblk and xennet).
     As far as why they are modules to begin with; I'm not 100% sure, but the policy tends to be to make everything as modular as possible.  This cuts down on the size of the kernel, which is, in fact, a concern.
     Anyway, I hope that helps, and I'm glad you got it working.

Chris Lalancette