[Crash-utility] [PATCH v5 0/4] arm64: more improvement of bt -f
Dave Anderson
anderson at redhat.com
Wed Jun 29 20:09:01 UTC 2016
Hi Takahiro,
I applied patches 1/2 and 2/2 from the v5 patchset. But I can't
believe the results are what you intended?
For example, taking the 4.6 vmcore that you gave to me, here is the
current crash utility's output of "bt -a", where the crashing task
entered crash_kexec() via the sysrq-c page fault exception, and the
tasks on the other cpus have all entered crash_save_cpu() on their
IRQ stack as a result of the shutdown IPI, one from user-space and
the others from the kernel:
crash> bt -a
PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
#0 [ffff800022f42e50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022f43010] handle_IPI at ffff00000808e718
#2 [ffff800022f43040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022f43080] el1_irq at ffff000008084720
--- <IRQ stack> ---
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff000008dc3f10 PSTATE: 60400149
X29: ffff000008dc3f10 X28: ffff000008dc0000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff000008dc0000 X22: ffff000008bd0270 X21: ffff000008dc0000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 00000000a632f641
X17: 0000ffff7da57880 X16: ffff0000081d9838 X15: 00000000383a0a79
X14: 00000000b2b0b162 X13: 000000005f2cbeec X12: 0000000000045a9e
X11: ffff8000213bd800 X10: 0000000000000850 X9: ffff000008dc0000
X8: 000000010000aa07 X7: 000000000000003d X6: 0015752a00000000
X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
#4 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857c0
#5 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26cc
#6 [ffff000008dc3f80] rest_init at ffff0000087c7930
#7 [ffff000008dc3fa0] start_kernel at ffff000008b10b70
PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
#0 [ffff800022f56e50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022f57010] handle_IPI at ffff00000808e718
#2 [ffff800022f57040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022f57080] el1_irq at ffff000008084720
--- <IRQ stack> ---
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218cff60 PSTATE: 60000349
X29: ffff8000218cff60 X28: ffff8000218cc000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218cc000 X22: ffff000008bd0270 X21: ffff8000218cc000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000005
X14: 0000000c80096000 X13: ffff8000212b6600 X12: 0000000000047ae2
X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218cc000
X8: 000000010000aa07 X7: 0000000000000243 X6: 0015752a00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
#4 [ffff8000218cff60] arch_cpu_idle at ffff0000080857c0
#5 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26cc
#6 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1ec
PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
#0 [ffff800022f6ae50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022f6b010] handle_IPI at ffff00000808e718
#2 [ffff800022f6b040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022f6b080] el0_irq_naked at ffff000008084c4c
--- <IRQ stack> ---
PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE: 20000000
PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
#0 [ffff800020b0ba70] crash_kexec at ffff00000812b0ac
#1 [ffff800020b0ba90] die at ffff000008088ce8
#2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fac
#3 [ffff800020b0bb00] do_page_fault at ffff000008096814
#4 [ffff800020b0bb60] do_translation_fault at ffff00000809690c
#5 [ffff800020b0bb70] do_mem_abort at ffff00000808128c
#6 [ffff800020b0bd40] el1_da at ffff000008084568
PC: ffff000008457fc8 [sysrq_handle_crash+32]
LR: ffff000008457fb4 [sysrq_handle_crash+12]
SP: ffff800020b0bd40 PSTATE: 60400149
X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
ORIG_X0: ffff000008dda000 SYSCALLNO: ffff80002104d418
#7 [ffff800020b0bd50] __handle_sysrq at ffff000008458644
#8 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458ac0
#9 [ffff800020b0bdb0] proc_reg_write at ffff00000822541c
#10 [ffff800020b0bdd0] __vfs_write at ffff0000081c4774
#11 [ffff800020b0be50] vfs_write at ffff0000081c5588
#12 [ffff800020b0be90] sys_write at ffff0000081c68fc
#13 [ffff800020b0bed0] el0_svc_naked at ffff000008084e70
PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
#0 [ffff800022f92e50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022f93010] handle_IPI at ffff00000808e718
#2 [ffff800022f93040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022f93080] el1_irq at ffff000008084720
--- <IRQ stack> ---
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218dbf60 PSTATE: 60000349
X29: ffff8000218dbf60 X28: ffff8000218d8000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218d8000 X22: ffff000008bd0270 X21: ffff8000218d8000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000000
X14: 0000000000000000 X13: 00000000000003e7 X12: 0000000000000016
X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218d8000
X8: ffff8000218c3ab0 X7: 7fffffffffffffff X6: 0015752a00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
#4 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857c0
#5 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26cc
#6 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1ec
PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
#0 [ffff800022fa6e50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022fa7010] handle_IPI at ffff00000808e718
#2 [ffff800022fa7040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022fa7080] el1_irq at ffff000008084720
--- <IRQ stack> ---
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218dff60 PSTATE: 60000349
X29: ffff8000218dff60 X28: ffff8000218dc000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218dc000 X22: ffff000008bd0270 X21: ffff8000218dc000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000007
X14: 0000000000000000 X13: 000000000000039d X12: 0000000000000000
X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218dc000
X8: 000000010000a479 X7: 7fffffffffffffff X6: 0016694e00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
#4 [ffff8000218dff60] arch_cpu_idle at ffff0000080857c0
#5 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26cc
#6 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1ec
PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
#0 [ffff800022fbae50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022fbb010] handle_IPI at ffff00000808e718
#2 [ffff800022fbb040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022fbb080] el1_irq at ffff000008084720
--- <IRQ stack> ---
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218ebf60 PSTATE: 60000349
X29: ffff8000218ebf60 X28: ffff8000218e8000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218e8000 X22: ffff000008bd0270 X21: ffff8000218e8000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 000000000000000a
X14: 0000000000000000 X13: 00000000000003a1 X12: 0000000000000000
X11: ffff8000213a0d00 X10: 0000000000000850 X9: ffff8000218e8000
X8: ffff8000218c53b0 X7: 00000000000001c4 X6: 002887fa00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
#4 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857c0
#5 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26cc
#6 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1ec
PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
#0 [ffff800022fcee50] crash_save_cpu at ffff00000812ae44
#1 [ffff800022fcf010] handle_IPI at ffff00000808e718
#2 [ffff800022fcf040] gic_handle_irq at ffff0000080815f8
#3 [ffff800022fcf080] el1_irq at ffff000008084720
--- <IRQ stack> ---
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218eff60 PSTATE: 60400149
X29: ffff8000218eff60 X28: ffff8000218ec000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218ec000 X22: ffff000008bd0270 X21: ffff8000218ec000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: fffffffffffffffb
X17: 0000ffff9dc8ba48 X16: ffff0000081da048 X15: 003b9aca00000000
X14: 0000000000000000 X13: 0000000000000001 X12: 0000000000000000
X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218ec000
X8: 000000010000a280 X7: ffff8000209b39c8 X6: ffff8000209b39c9
X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
#4 [ffff8000218eff60] arch_cpu_idle at ffff0000080857c0
#5 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26cc
#6 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1ec
crash>
And that's precisely what we want to see. (other than perhaps the
in-kernel exception frame's dump of the ORIG_X0 and SYSCALL registers)
Now, with v5's patches 1/2 and 2/2 patches applied, this is what gets displayed:
crash> bt -a
PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
#0 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857c0
#1 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26cc
#2 [ffff000008dc3f80] rest_init at ffff0000087c7930
#3 [ffff000008dc3fa0] start_kernel at ffff000008b10b70
PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
#0 [ffff8000218cff60] arch_cpu_idle at ffff0000080857c0
#1 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26cc
#2 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1ec
PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE: 20000000
#0 [user space]
PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
#0 [ffff800020b0ba70] crash_kexec at ffff00000812b0ac
#1 [ffff800020b0ba90] die at ffff000008088ce8
#2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fac
#3 [ffff800020b0bb00] do_page_fault at ffff000008096814
#4 [ffff800020b0bb60] do_translation_fault at ffff00000809690c
#5 [ffff800020b0bb70] do_mem_abort at ffff00000808128c
#6 [ffff800020b0bc20] el1_da at ffff000008084568
--- <Exception in kernel> ---
PC: ffff000008457fc8 [sysrq_handle_crash+32]
LR: ffff000008457fb4 [sysrq_handle_crash+12]
SP: ffff800020b0bd40 PSTATE: 60400149
X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
#7 [ffff800020b0bd40] sysrq_handle_crash at ffff000008457fc8
#8 [ffff800020b0bd50] __handle_sysrq at ffff000008458644
#9 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458ac0
#10 [ffff800020b0bdb0] proc_reg_write at ffff00000822541c
#11 [ffff800020b0bdd0] __vfs_write at ffff0000081c4774
#12 [ffff800020b0be50] vfs_write at ffff0000081c5588
#13 [ffff800020b0be90] sys_write at ffff0000081c68fc
#14 [ffff800020b0bed0] el0_svc_naked at ffff000008084e70
--- <Exception in user> ---
PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
#0 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857c0
#1 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26cc
#2 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1ec
PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
#0 [ffff8000218dff60] arch_cpu_idle at ffff0000080857c0
#1 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26cc
#2 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1ec
PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
#0 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857c0
#1 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26cc
#2 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1ec
PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
#0 [ffff8000218eff60] arch_cpu_idle at ffff0000080857c0
#1 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26cc
#2 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1ec
crash>
What happened? This can't possibly be what you intended, is it?
Dave
More information about the Crash-utility
mailing list