[libvirt] [BUG] Xen x86_64 needs PAE?

[Daniel P. Berrange]

On Mon, Aug 09, 2010 at 02:20:42PM +0200, Philipp Hahn wrote:
> Hello,
> 
> I tried to install "Microsoft Windows 2008 Server R2" on an Amd64 host using 
> Xen-3.4.3. The installation aborted with the following error message:
> > Windows Boot Mager
> ...
> > File: \windows\system32\boot\winload.exe
> > Status: 0xc000035a
> > Info: Attempting to load a 64-bit application, however this CPU is not 
> compatible with 64-bit mode.
> 
> For testing I converted the libvirt-XML file to an Xen-Xm file using "virsh 
> domxml-to-native" and started that file with "xm create", which worked.
> Comparing the output of "xm list -a" of the broken libvirt- and the working 
> xm-configuration I found several differences, especially with "pae": 
> Using "xm create" pae is set to 1, while with libvirt pae is set to 0. After 
> adding the /domain/featues/pae node to libvirts XML domain file, Windows 
> installed fine.

If you can provide the 'xm' config file, the libvirt XML and the output
of 'xm list --list $GUESTNAME' when the guest is succesfully running we
can add a test case to validate this PAE setup & fix the xen driver.

> 
> Looking at "virsh capabilities" I get the following (abbreviated) output:
> > <capabilities>
> >   <guest>
> >     <os_type>hvm</os_type>
> >     <arch name='i686'>
> >       <wordsize>32</wordsize>
> ...
> >     </arch>
> >     <features>
> >       <pae/>
> >       <nonpae/>
> >       <acpi default='on' toggle='yes'/>
> >       <apic default='on' toggle='yes'/>
> >     </features>
> >   </guest>
> >   <guest>
> >     <os_type>hvm</os_type>
> >     <arch name='x86_64'>
> >       <wordsize>64</wordsize>
> ...
> >     </arch>
> >     <features>
> >       <acpi default='on' toggle='yes'/>
> >       <apic default='on' toggle='yes'/>
> >     </features>
> >   </guest>
> > </capabilities>
> 
> As you can see, neither "<pae/>" nor "<nopae/>" is listed for for the 64 bit 
> Xen-Fv domain type, which makes it hard to automatically create a correct 
> _and_ _working_ XML file for Xen-Fv-64 bit domains
> 
> Looking at "/sys/hypervisor/properties/capabilities" I find the following 
> capabilities reported by Xen
> > xen-3.0-x86_64
> > xen-3.0-x86_32p
> > hvm-3.0-x86_32
> > hvm-3.0-x86_32p
> > hvm-3.0-x86_64    
> 
> There's also no "hvm-3.0-x86_64p" which makes me wonder, if "<pae/>" is the 
> default for Xen-64 bit domains.
> 
> If that is true, should libvirt add "<pae/"> by default to the capabilities 
> report, or should libvirt silently pass "(pae 1)" when pushing the XML domain 
> description to Xend? Lookign at 

In 32-bit Xen, the pae flag indicates whether this is a 32bit PAE or
non-PAE guest. 

In 64-bit Xen, the pae flag indicates whether the CPU exposed supports
long mode. So if you don't set the PAE flag on 64-bit Xen, you get a
32-bit PAE guest. If you do set the PAE flag, you get a 64-bit guest.
Yes, you read that right, not setting PAE, gets you a PAE enabled guest
just with 32 instead of 64 bits :-)

In terms of the domain XML design, the choice between 32 and 64 bits is
intended to be done based on the arch field:

  <os>
    <type arch='x86_64'...
    ...
  </os>

The pae flag in domain XML should be reflecting whether pae is enabled
in the guest.

The capabilities XML should be reporting whether the guest ABI includes
the PAE flag or not. So I think the 32bit capabilities shouldn't have
included <nonpae/> there, and perhaps 64-bit should have had <pae/>. 
Ultimately though I think our mistake is in passing the <pae> flag from
the domain XML, straight through to the hypervisor. I'm rather wary of
changing this, because of the risk of regressions, but if you can supply
the 3 files I asked for above, we can investigate whether its feasible

Daniel
-- 
|: Red Hat, Engineering, London    -o-   http://people.redhat.com/berrange/ :|
|: http://libvirt.org -o- http://virt-manager.org -o- http://deltacloud.org :|
|: http://autobuild.org        -o-         http://search.cpan.org/~danberr/ :|
|: GnuPG: 7D3B9505  -o-   F3C9 553F A1DA 4AC2 5648 23C1 B3DF F742 7D3B 9505 :|