[fedora-virt] Fedora 10 and Fedora 11 Xen 32b domUs failing on x86_64 el5.3 dom0

Mon Jul 27 22:11:11 UTC 2009

On Mon, Jul 27, 2009 at 10:45:06PM +0100, Daniel P. Berrange wrote:
> On Tue, Jul 28, 2009 at 12:39:17AM +0300, Pasi K?rkk?inen wrote:
> > On Tue, Mar 24, 2009 at 07:28:46PM +0000, Daniel P. Berrange wrote:
> > > On Tue, Mar 24, 2009 at 09:25:02PM +0200, Pasi K?rkk?inen wrote:
> > > > Hello!
> > > > 
> > > > I've used a lot of time trying to get my custom Xen pv_ops dom0 kernel working with
> > > > virt-install and/or virt-manager on Fedora 10, and now it seems I got things
> > > > working. 
> > > > 
> > > > If you want to play with this you need:
> > > > 
> > > > 1) New enough pv_ops dom0 kernel (2.6.29-rc8 or newer) so it has /sys/hypervisor support included
> > > > 	- Compile with CONFIG_HIGHPTE=n since it seems to be broken still
> > > > 2) libvirt 0.6.1 and related packages from Fedora 10 updates-testing
> > > > 
> > > > In addition to those I'm using Xen 3.3.1-9 packages from rawhide/F11 rebuilt for F10. 
> > > > 
> > > > With the older Fedora 10 libvirt packages libvirtd was crashing often for me, and
> > > > I had some other issues with virt-install console window not opening but stalling etc..
> > > > 
> > > > Today I was able to run the following on Fedora 10 32bit PAE pv_ops dom0:
> > > > 
> > > > - CentOS 5.3 32bit PAE PV domU
> > > > - Fedora 10 32bit PAE PV domU 
> > > > - Use virt-install to install Fedora 10 32bit PAE PV domU (using custom kickstart
> > > > to force PAE kernel installation to avoid the anaconda BUG which installs
> > > > wrong non-PAE kernel as a default).
> > > > 
> > > > Fedora 11 (rawhide) installation most probably works too. 
> > > 
> > > FYI, I've just testing i686 and x86_64  Fedora 11 guest installs on a 
> > > RHEL5 Xen host and they both work flawlessly out of the box - including
> > > a  sane mouse pointer running in absolute, so no dual-cursor problems,
> > > and installing correect PAE kernels
> > > 
> > 
> > Replying to old thread.. today I tried installing F11 i386 domU with
> > virt-install on centos 5.3 x86_64 dom0, and it doesn't seem to work..
> > 
> > The graphical VNC console doesn't show up, and it seems the domU kernel
> > crashes.
> > 
> > x86_64 domU installation on x86_64 dom0 seems to work though.
> > Are you sure i386 on x86_64 worked for you? 
> 
> Yep, just checked my RHEL-5 box and it has this config
> 
> name = "f11i686xen"
> uuid = "1417ac4b-bd9a-61cc-6faf-8e842cd72cb9"
> maxmem = 900
> memory = 500
> vcpus = 1
> bootloader = "/usr/bin/pygrub"
> on_poweroff = "destroy"
> on_reboot = "restart"
> on_crash = "restart"
> vfb = [ "type=vnc,vncunused=1" ]
> disk = [ "tap:aio:/var/lib/xen/images/f11i686.img,xvda,w" ]
> vif = [ "mac=00:16:3e:71:38:63,bridge=xenbr0" ]
> 
> 

Nothing weird here.. My guest has 1 vcpu aswell, and I'm using vnc.

> And the guest disk is fully instlaled
> 
> # ls -lhs /var/lib/xen/images/f11i686.img
> 3.4G -rwxr-xr-x 1 root root 5.1G Jun  4 06:33 /var/lib/xen/images/f11i686.img
> 
> 
> FWIW, the guest kernel here is 2.6.29-0.258.rc8.git2.fc11.i686.PAE
> 
> So its possible things have changed since I did my install .... 
> 

Default kernel in Fedora 11 is: kernel-PAE-2.6.29.4-167.fc11.i686.rpm

> Also my x86_64 host only has 2 GB of RAM, so if there's any bugs with
> i686 guests at the >4GB phys RAM mark I wouldn't have seen them
> 

My guests were using 512 MB and 1024 MB of RAM..

I tried again now, and yes, F10 and F11 i386 guests fail on CentOS 5.3
x86_64 dom0 with domU kernel crashes like this:

http://pasik.reaktio.net/fedora/fedora10-32b-xen-domu-on-centos53-64b-dom0-install-crash.txt
http://pasik.reaktio.net/fedora/fedora11-32b-xen-domu-on-centos53-64b-dom0-crash2.txt

Checking if this processor honours the WP bit even in supervisor mode...Ok.
1 multicall(s) failed: cpu 0
Pid: 0, comm: swapper Not tainted 2.6.29.4-167.fc11.i686.PAE #1
Call Trace:
 [<c0714be7>] ? printk+0x14/0x1d
 [<c0405020>] xen_mc_flush+0x12d/0x1f9
 [<c0405880>] xen_mc_issue+0x19/0x4d
 [<c0406293>] xen_set_pud_hyper+0x79/0x81
 [<c040632a>] xen_set_pud+0x8f/0x95
 [<c04220e0>] zap_low_mappings+0x2d/0x42
 [<c091c917>] mem_init+0x2a0/0x2a8
 [<c0906828>] start_kernel+0x264/0x313
 [<c090609d>] i386_start_kernel+0x8c/0x97
 [<c090c351>] xen_start_kernel+0x7f9/0x802
------------[ cut here ]------------
kernel BUG at arch/x86/xen/multicalls.c:180!   
invalid opcode: 0000 [#1] SMP

Hmm.. I wonder what's causing this.

Can you test if it works for you?

I used virt-install like this:
virt-install --debug -n testvm -r 512 --vcpus=1 -f /dev/VolGroup00/testvm_disk --vnc -p -l "ftp://ftp.funet.fi/pub/mirrors/fedora.redhat.com/pub/fedora/linux/releases/11/Fedora/i386/os"

The graphical VNC console newer shows up, and when I run "xm console testvm"
I see that kernel crash.

-- Pasi