[Crash-utility] Xen stack traces using crash ...

[Steve Ofsthun]

I'm having trouble examining xen stack traces using crash.  I have a full xen dump image and can examine the stack traces for any active vcpu.  But when I try to trace a non-active vcpu, I seem to get a copy of the stack trace for the last pcpu the non-active vcpu ran on.

For example:

crash> vcpus
   VCID  PCID       VCPU       ST T DOMID      DOMAIN
      0     0 ffff8300dfdfc080 RU I 32767 ffff8300dc250080
>     1     1 ffff8300dbe02080 RU I 32767 ffff8300dc250080
>     2     2 ffff8300dc24e080 RU I 32767 ffff8300dc250080
>     3     3 ffff8300dfdfa080 RU I 32767 ffff8300dc250080
>     4     4 ffff8300dbe06080 RU I 32767 ffff8300dc250080
>     5     5 ffff8300dbefa080 RU I 32767 ffff8300dc250080
>     6     6 ffff8300dc27e080 RU I 32767 ffff8300dc250080
>     7     7 ffff8300dc27a080 RU I 32767 ffff8300dc250080
>     8     8 ffff8300dc266080 RU I 32767 ffff8300dc250080
>     9     9 ffff8300dc262080 RU I 32767 ffff8300dc250080
>    10    10 ffff8300dfdcc080 RU I 32767 ffff8300dc250080
>*   11    11 ffff8300dfdc8080 RU I 32767 ffff8300dc250080
>    12    12 ffff8300dbe34080 RU I 32767 ffff8300dc250080
>    13    13 ffff8300dbe30080 RU I 32767 ffff8300dc250080
>    14    14 ffff8300dbedc080 RU I 32767 ffff8300dc250080
>    15    15 ffff8300dbed8080 RU I 32767 ffff8300dc250080
>     0     0 ffff8300dfd80080 RU 0     0 ffff8300dfd90080
      1     1 ffff8300dbe7e080 BL 0     0 ffff8300dfd90080
      2     2 ffff8300dbe7c080 BL 0     0 ffff8300dfd90080
      3     3 ffff8300dbe7a080 BL 0     0 ffff8300dfd90080
      4     4 ffff8300dbe78080 BL 0     0 ffff8300dfd90080
      5     5 ffff8300dbe74080 BL 0     0 ffff8300dfd90080
      6     6 ffff8300dbe72080 BL 0     0 ffff8300dfd90080
      7     7 ffff8300dbe70080 BL 0     0 ffff8300dfd90080
      8     8 ffff8300dbe6e080 BL 0     0 ffff8300dfd90080
      9     9 ffff8300dbe6c080 BL 0     0 ffff8300dfd90080
     10    10 ffff8300dbe6a080 BL 0     0 ffff8300dfd90080
     11    11 ffff8300dbe68080 BL 0     0 ffff8300dfd90080
     12    12 ffff8300dbe66080 BL 0     0 ffff8300dfd90080
     13    13 ffff8300dbe64080 BL 0     0 ffff8300dfd90080
     14    14 ffff8300dbe62080 BL 0     0 ffff8300dfd90080
     15    15 ffff8300dbe60080 BL 0     0 ffff8300dfd90080
      0    11 ffff8300dbe52080 BL U     7 ffff8300dbfc0080
crash> bt 11
PCPU: 11  VCPU: ffff8300dfdc8080
 #0 [ffff8300dbe3fbe0] kexec_crash at ffff828c8010cd8c
 #1 [ffff8300dbe3fc00] panic at ffff828c801205fe
 #2 [ffff8300dbe3fcc0] show_stack at ffff828c8013c1f3
 #3 [ffff8300dbe3fcf0] do_general_protection at ffff828c8013f07e
 #4 [ffff8300dbe3fd10] vpic_get_highest_priority_irq at ffff828c8015a2a3
 #5 [ffff8300dbe3fd48] svm_asid_inc_generation at ffff828c8015c95e
 #6 [ffff8300dbe3fd70] vpic_intercept_pic_io at ffff828c8015ae9e
 #7 [ffff8300dbe3fd98] __context_switch at ffff828c8012570e
 #8 [ffff8300dbe3fe20] handle_exception_saved at ffff828c801884be
 #9 [ffff8300dbe3fea8] restore_all_xen at ffff828c8018809b
#10 [ffff8300dbe3fed8] idle_loop at ffff828c80125c2f
crash> 

This is the vcpu that crashed Xen.

Now I want to examine the vcpu for domain 7, since I know that vcpu triggered the fatal crash.

crash> bt ffff8300dbe52080
PCPU: 11  VCPU: ffff8300dbe52080
 #0 [ffff8300dbe3fbe0] kexec_crash at ffff828c8010cd8c
 #1 [ffff8300dbe3fc00] panic at ffff828c801205fe
 #2 [ffff8300dbe3fcc0] show_stack at ffff828c8013c1f3
 #3 [ffff8300dbe3fcf0] do_general_protection at ffff828c8013f07e
 #4 [ffff8300dbe3fd10] vpic_get_highest_priority_irq at ffff828c8015a2a3
 #5 [ffff8300dbe3fd48] svm_asid_inc_generation at ffff828c8015c95e
 #6 [ffff8300dbe3fd70] vpic_intercept_pic_io at ffff828c8015ae9e
 #7 [ffff8300dbe3fd98] __context_switch at ffff828c8012570e
 #8 [ffff8300dbe3fe20] handle_exception_saved at ffff828c801884be
 #9 [ffff8300dbe3fea8] restore_all_xen at ffff828c8018809b
#10 [ffff8300dbe3fed8] idle_loop at ffff828c80125c2f
crash> 

This trace is the same stack page as the idle vcpu running on pcpu 11.

Any ideas?

Thanks,
Steve