[Fedora-xen] f10 x86_64 xen VM guests fail to boot on f8 host
Virtualization
virtualization at webwombat.com.au
Tue Jan 20 06:18:48 UTC 2009
Hi list,
>From the Intel® Virtualization Technology Specification
for the IA-32 Intel® Architecture (2005):
"2.9.2 Information for VM Exits Due to Vectored Events
Event-specific information is provided for VM exits due to the following vectored events:
exceptions (including those generated by the instructions INT3, INTO, BOUND, and UD2);
external interrupts that occur while the “acknowledge interrupt on exit” VM-exit control is 1;
and non-maskable interrupts (NMIs). This information is provided in the following fields:" ....
The <0f> 0b in the "Code:" section are the UD2 instruction.
Checking through the OpCode map for the Xeon processor, this is an
invalid op code. In VT processors the software guide indicates that a
program can communicate various events and state information to the
underlying virtualization supervisor by executing a UD2 (and some others
ops like it).
I think that in a non-VT cpu it's actually a "real" invalid op code. The
stuff (hardware) which flips over to the supervisor with all the needed
info from the virtual machine isn't there.
KVM uses this, from the patches I've seen Googling around for UD2 (if I
understand correctly).
So why a UD2 in the code? It's highly unlikely that it's just some
random bytes that happen to be a UD2. Possibly the kernel thinks it's in
fully virt mode at some point? The image notes do seem to indicate this.
Cheers
Phill.
On Tue, 2009-01-20 at 14:01 +0900, Jon Swanson wrote:
> Hi Mark, thank you very much for your help.
>
> I took an f10host which boots on MachineA and copied it to MachineB,
> modified the config to include on_crash=preserve, and booted it with xm
> create.
> ------------------------------------------------------------------------
> ------------------------------------------------
> xenctx output:
> /usr/lib64/xen/bin/xenctx -s System.map-2.6.27.5-117.fc10.x86_64 46
> rip: ffffffff8100b8a2 set_page_prot+0x6d
> rsp: ffffffff81573f08
> rax: ffffffea rbx: 000016e1 rcx: 00000055 rdx: 00000000
> rsi: 800000014ffc6061 rdi: ffffffff816e1000 rbp: ffffffff81573f68
> r8: 0000000f r9: ffffffff817eb450 r10: ffffffff817eb650 r11:
> 00000010
> r12: ffffffff816e1000 r13: 800000014ffc6061 r14: 8000000000000161
> r15: 00000016
> cs: 0000e033 ds: 00000000 fs: 00000000 gs: 00000000
>
> Stack:
> 0000000000000055 0000000000000010 ffffffff8100b8a2 000000010000e030
> 0000000000010082 ffffffff81573f48 000000000000e02b ffffffff8100b89e
> 0000000000000200 ffffffff816e4000 0000000000000800 0000000000002c00
> ffffffff81573ff8 ffffffff815a3c60 0000000000002c00 0000000000000000
>
> Code:
> 7b 4a 1d 00 4c 89 e7 4c 89 ee 31 d2 e8 22 d9 ff ff 85 c0 74 04 <0f> 0b
> eb fe 5b 41 5c 41 5d 41 5e
>
> Call Trace:
> [<ffffffff8100b8a2>] set_page_prot+0x6d <--
> [<ffffffff8100b8a2>] set_page_prot+0x6d
> [<ffffffff8100b89e>] set_page_prot+0x69
> [<ffffffff815a3c60>] xen_start_kernel+0x5dd
>
>
> ------------------------------------------------------------------------
> ------------------------------------------------
> Dmesg also has something which may make sense to someone wiser than
> myself. Specifically:
> (XEN) traps.c:405:d46 Unhandled invalid opcode fault/trap [#6] in domain
> 46 on VCPU 0 [ec=0000]
>
>
> xm dmesg
> ....
> (XEN) ffffffff82035000 ffffffff82036000 ffffffff82037000
> ffffffff82038000
> (XEN) mm.c:1362:d46 Bad L1 flags 800000
> (XEN) traps.c:405:d46 Unhandled invalid opcode fault/trap [#6] in domain
> 46 on VCPU 0 [ec=0000]
> (XEN) domain_crash_sync called from entry.S
> (XEN) Domain 46 (vcpu#0) crashed on cpu#2:
> (XEN) ----[ Xen-3.1.4 x86_64 debug=n Not tainted ]----
> (XEN) CPU: 2
> (XEN) RIP: e033:[<ffffffff8100b8a2>]
> (XEN) RFLAGS: 0000000000000282 CONTEXT: guest
> (XEN) rax: 00000000ffffffea rbx: 00000000000016e1 rcx:
> 0000000000000055
> (XEN) rdx: 0000000000000000 rsi: 800000014ffc6061 rdi:
> ffffffff816e1000
> (XEN) rbp: ffffffff81573f68 rsp: ffffffff81573f08 r8:
> 000000000000000f
> (XEN) r9: ffffffff817eb450 r10: ffffffff817eb650 r11:
> 0000000000000010
> (XEN) r12: ffffffff816e1000 r13: 800000014ffc6061 r14:
> 8000000000000161
> (XEN) r15: 0000000000000016 cr0: 000000008005003b cr4:
> 00000000000006f0
> (XEN) cr3: 0000000144f18000 cr2: 0000000000000000
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: e02b cs: e033
> (XEN) Guest stack trace from rsp=ffffffff81573f08:
> (XEN) 0000000000000055 0000000000000010 ffffffff8100b8a2
> 000000010000e030
> (XEN) 0000000000010082 ffffffff81573f48 000000000000e02b
> ffffffff8100b89e
> (XEN) 0000000000000200 ffffffff816e4000 0000000000000800
> 0000000000002c00
> (XEN) ffffffff81573ff8 ffffffff815a3c60 0000000000002c00
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> ffffffff8208b000
> (XEN) 0000000000010000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> ffffffff82008000
> (XEN) ffffffff82009000 ffffffff8200a000 ffffffff8200b000
> ffffffff8200c000
> (XEN) ffffffff8200d000 ffffffff8200e000 ffffffff8200f000
> ffffffff82010000
> (XEN) ffffffff82011000 ffffffff82012000 ffffffff82013000
> ffffffff82014000
> (XEN) ffffffff82015000 ffffffff82016000 ffffffff82017000
> ffffffff82018000
> (XEN) ffffffff82019000 ffffffff8201a000 ffffffff8201b000
> ffffffff8201c000
> (XEN) ffffffff8201d000 ffffffff8201e000 ffffffff8201f000
> ffffffff82020000
> (XEN) ffffffff82021000 ffffffff82022000 ffffffff82023000
> ffffffff82024000
> (XEN) ffffffff82025000 ffffffff82026000 ffffffff82027000
> ffffffff82028000
> (XEN) ffffffff82029000 ffffffff8202a000 ffffffff8202b000
> ffffffff8202c000
> (XEN) ffffffff8202d000 ffffffff8202e000 ffffffff8202f000
> ffffffff82030000
> (XEN) ffffffff82031000 ffffffff82032000 ffffffff82033000
> ffffffff82034000
> (XEN) ffffffff82035000 ffffffff82036000 ffffffff82037000
> ffffffff82038000
> ...
>
> ------------------------------------------------------------------------
> ------------------------------------------------
>
>
> -----Original Message-----
> From: Mark McLoughlin [mailto:markmc at redhat.com]
> Sent: Friday, January 16, 2009 7:34 PM
> To: Jon Swanson
> Cc: fedora-xen at redhat.com
> Subject: Re: [Fedora-xen] f10 x86_64 xen VM guests fail to boot on f8
> host
>
> Hi Jon,
>
> On Fri, 2009-01-16 at 18:02 +0900, Jon Swanson wrote:
> > This is a cross post of the same subject on the Fedora Forums. If
> > this is bad practice let me know and i'll never do it again.
>
> Mailing lists can often be a better way to get help from developers, so
> posting here is no problem.
>
> Also, fedora-virt at redhat.com might be a better place to post questions
> these days - it's not clear whether the fedora-xen list has a future.
>
> > Additional log info is available at
> > http://forums.fedoraforum.org/showthread.php?p=1149972&posted=1#post11
> > 49
> > 972
> >
> > I have two machines running fresh installs of f8 with the xen. Kernel
> > and all software versions are the same on both.
>
> You've seen this then, right?
>
>
> http://fedoraproject.org/wiki/Bugs/F10Common#Installing_Fedora_10_DomU_o
> n_Fedora_8_Dom0_Fails
>
> > /var/log/xen/xend.log relevant output:
> >
> > [2009-01-16 14:45:32 4120] DEBUG (DevController:150) Waiting for
> > devices vtpm.
> > [2009-01-16 14:45:32 4120] INFO (XendDomain:1130) Domain f10testB (21)
>
> > unpaused.
> > [2009-01-16 14:45:32 4120] WARNING (XendDomainInfo:1203) Domain has
> > crashed: name=f10testB id=21.
> > [2009-01-16 14:45:32 4120] DEBUG (XendDomainInfo:1802)
> > XendDomainInfo.destroy: domid=21
> > [2009-01-16 14:45:32 4120] DEBUG (XendDomainInfo:1821)
> > XendDomainInfo.destroyDomain(21)
> > I've also tried moving a functional guest from MachineA to MachineB to
>
> > boot it there, with the same results. Guest will not boot on MachineB.
> >
> > f8 64bit guests will boot on MachineB with no problems.
> > f10 32bit guests will boot on MachineB with no problems.
> >
> > Only 64bit machines seem to be borked.
>
> Okay, sounds like it might "just" be a F10 kernel bug.
>
> Try doing this to get a stack trace:
>
> 1) Set "on_crash=preserve" in your domain config
>
> 2) Copy the guest kernel's System.map to the host
>
> 2) Once the guest has crashed, run:
>
> /usr/lib/xen/bin/xenctx -s System.map <domid>
>
> Cheers,
> Mark.
>
>
>
> --
> Fedora-xen mailing list
> Fedora-xen at redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-xen
>
More information about the Fedora-xen
mailing list