<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
</head>
<body>
<div style="word-wrap:break-word">
<div>Hi, Eduardo:</div>
<div><br>
</div>
<div>Thanks a lot for the comments! It is really helpful! Based on your suggestion, I did a quick verification on the cpu flags, and the result is very ugly…KVM crashes pretty much for most of the flags I tested:</div>
<div><br>
</div>
<div>No crash: fxsr</div>
<div>Crash: sse2, sse, mmx, clflush, pse36, pat, cmov, mca</div>
<div><br>
</div>
<div>The test was conducted by using qemu-kvm-rhev-0.12.1.2-2.351.el6.x86_64. I did yum update last night going to bed and the qemu-kvm-0.12.1.2-2.335.el6.x86_64 was obsoleted by qemu-kvm-rhev-0.12.1.2-2.351.el6.x86_64.</div>
<div><br>
</div>
<div>I didn't exhaust the list, but all of these flags should be supported by Nehalem. At this moment, do you think we may have CPU defect? Please see attached TXT file for details.</div>
<div><br>
</div>
<div>Thanks!</div>
<div><br>
</div>
<div>Shixiong</div>
<div><br>
</div>
<div><br>
</div>
<div></div>
</div>
<div style="word-wrap:break-word">
<div></div>
<div><br>
</div>
<br>
<div>
<div>
<div style="color:rgb(0,0,0); font-family:Helvetica; font-size:medium; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; line-height:normal; orphans:2; text-indent:0px; text-transform:none; white-space:normal; widows:2; word-spacing:0px">
</div>
<div style="color:rgb(0,0,0); font-family:Helvetica; font-size:medium; font-style:normal; font-variant:normal; font-weight:normal; letter-spacing:normal; line-height:normal; orphans:2; text-indent:0px; text-transform:none; white-space:normal; widows:2; word-spacing:0px">
<div style="margin:0in 0in 0.0001pt; font-size:12pt; font-family:'Times New Roman',serif">
<span style="font-size:13.5pt; font-family:Calibri,sans-serif"><br>
</span></div>
</div>
</div>
</div>
<div><br>
</div>
<div><br>
</div>
<br>
<div>
<div>On Feb 15, 2013, at 10:52 PM, Eduardo Habkost <<a href="mailto:ehabkost@redhat.com">ehabkost@redhat.com</a>></div>
<div> wrote:</div>
<br class="x_Apple-interchange-newline">
<blockquote type="cite"><br>
Hi, all,<br>
<br>
I'm Eduardo from the KVM team. Some comments and questions below:<br>
<br>
<br>
On Sat, Feb 16, 2013 at 02:31:24AM +0000, Shixiong Shang (shshang) wrote:<br>
<blockquote type="cite">Hi, Perry and Karen:<br>
<br>
I did some further investigation tonight. The VM instance was<br>
initiated with lot of parameters, among which, here is one line<br>
related to CPU model:<br>
<br>
-cpu Nehalem,+rdtscp,+vmx,+ht,+ss,+acpi,+ds,+vme -enable-kvm<br>
<br>
<br>
Based on qemu-kvm command and cpu_map.xml file, Nehalem and all of the<br>
flags are supported. However, when I tried to perform CPU check, KVM<br>
crashed again. The backtrace is identical to the ones I saw in failed<br>
VM instance log:<br>
</blockquote>
<br>
The "check" parameter asks QEMU to print warnings if some CPU features<br>
are not supported by the host CPU, but QEMU will start the guest<br>
normally after that. So, if you got to the "VNC server running" stage,<br>
it means all CPU features from the QEMU "Nehalem" CPU model should be<br>
supported by your host CPU + kernel, and the crash happened while the<br>
guest was already running, not during the CPU feature check.<br>
<br>
<blockquote type="cite"><br>
<br>
[root@as-cmp1 libvirt]# /usr/libexec/qemu-kvm -cpu Nehalem,check<br>
</blockquote>
<br>
I am assuming you used just the above command with no extra parameters<br>
(meaning you don't even need a disk image to reproduce the bug), right?<br>
<br>
<br>
<blockquote type="cite">VNC server running on `::1:5900'<br>
KVM internal error. Suberror: 2<br>
</blockquote>
<br>
How long does the error message take to appear, after starting qemu-kvm?<br>
<br>
<blockquote type="cite">extra data[0]: 80000003<br>
extra data[1]: 80000603<br>
</blockquote>
<br>
The data above is weird: the CPU is reporting that it was trying to<br>
deliver an int3 (but with the interrupt type bits set to "external<br>
interrupt", which doesn't make sense), and got another int3 interrupt<br>
generated when trying to deliver it.<br>
<br>
It doesn't look right (the codes don't seem to make sense), and even if<br>
it was right, simply running qemu-kvm with no arguments shouldn't end up<br>
generating int3 interrupts at all.<br>
<br>
I would test this in other machines, to make sure this is really not a<br>
hardware defect. Could you send the contents of /proc/cpuinfo? If you<br>
are able to install the x86info package, the output of 'x86info -v -a'<br>
would be useful, too.<br>
<br>
<br>
<blockquote type="cite">rax 00000000000003c3 rbx 00000000000008f2 rcx 000000000000013f rdx 000000000000ffdf<br>
rsi 0000000000000006 rdi 000000000000c993 rsp 00000000000003aa rbp 000000000000f000<br>
r8 0000000000000000 r9 0000000000000000 r10 0000000000000000 r11 0000000000000000<br>
r12 0000000000000000 r13 0000000000000000 r14 0000000000000000 r15 0000000000000000<br>
rip 00000000000010e2 rflags 00000286<br>
</blockquote>
<br>
Interesting, RIP is different from your previous report. Does the value<br>
change if you run "/usr/libexec/qemu-kvm -cpu Nehalem,check" again?<br>
<br>
<br>
<blockquote type="cite">cs c000 (000c0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)<br>
ds c000 (000c0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)<br>
es f000 (000f0000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)<br>
ss 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)<br>
fs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)<br>
gs 0000 (00000000/0000ffff p 1 dpl 3 db 0 s 1 type 3 l 0 g 0 avl 0)<br>
tr 0000 (feffd000/00002088 p 1 dpl 0 db 0 s 0 type b l 0 g 0 avl 0)<br>
ldt 0000 (00000000/0000ffff p 1 dpl 0 db 0 s 0 type 2 l 0 g 0 avl 0)<br>
gdt fc558/37<br>
idt 0/3ff<br>
cr0 10 cr2 0 cr3 0 cr4 0 cr8 0 efer 0<br>
<br>
FYI, I am using this qemu-kvm version:<br>
qemu-kvm-0.12.1.2-2.335.el6.x86_64<br>
</blockquote>
<br>
Thanks. What are the versions of the kernel, seabios, vgabios, and gpxe<br>
packages?<br>
<br>
<blockquote type="cite"><br>
<br>
The potential workaround is to use generic CPU model, such as KVM64,<br>
with performance penalty. I will give it a try and keep you posted. In<br>
the meanwhile, if you can think of anything else, please let me at<br>
your early convenience.<br>
</blockquote>
<br>
If other CPU models work, it may simply indicate that some feature bit<br>
enabled by the Nehalem CPU model may be triggering the problem.<br>
<br>
If that's the case, one way to find out which feature is causing the<br>
problem is to try:<br>
<br>
$ /usr/lib/qemu-kvm -cpu qemu64,+sse2,+sse,+fxsr,+mmx,+clflush,+pse36,+pat,+cmov,+mca,+pge,+mtrr,+sep,+apic,+cx8,+mce,+pae,+msr,+tsc,+pse,+de,+fpu,+popcnt,+x2apic,+sse4.2,+sse4.1,+cx16,+ssse3,+sse3,+i64,+syscall,+xd,+lahf_lm,model=26<br>
<br>
I expect the bug to be reproduced easily using the above command-line.<br>
After that, you can gradually remove features from the command-line,<br>
until we find which one is triggering the problem.<br>
<br>
<br>
<blockquote type="cite"><br>
Thanks for your help!<br>
<br>
Shixiong<br>
<br>
<br>
</blockquote>
<br>
-- <br>
Eduardo<br>
</blockquote>
</div>
<br>
</div>
</body>
</html>