[Crash-utility] [PATCH v5 0/4] arm64: more improvement of bt -f
Dave Anderson
anderson at redhat.com
Wed Jun 29 20:44:41 UTC 2016
----- Original Message -----
>
>
> Hi Takahiro,
>
> I applied patches 1/2 and 2/2 from the v5 patchset. But I can't
> believe the results are what you intended?
Obviously I meant 1/4 and 2/4 above.
However, I was under the impression that the 3/4 patch was a standalone
patch that only served to change the text address displayed, and that
"adding this patch was a discussion topic":
> On arm64, the link register (LR) holds a return address, which is the one
> just after a branch instruction. So using a saved lr as PC for backtracing
> might cause some confusion.
> For example, in kernel/entry.S,
> work_resched:
> ...
> bl schedule
>
> ret_to_user:
> ...
>
> The current code shows "ret_o_user", instead of "work_resched",
> as a caller of schedule().
>
> This patch corrects a PC by decrementing it by 4.
> But please note that this change may also make people a bit confused
> because a value of LR in the stack dump of "bt -f" doesn't match with
> an address in one-line summary.
>
> #2 [ffffcc7511407eb0] schedule at ffff0000d628aee0
> ffffcc7511407eb0: ffffcc6d22f23080 ffff0000d5b44d6c <= LR
> ffffcc7511407ec0: ffffcc7511407ed0 0000000000000000
> #3 [ffffcc7511407ed0] work_resched at ffff0000d5b44d68 <= correcrted PC
>
> Signed-off-by: AKASHI Takahiro <takahiro akashi linaro org>
...and as you subsequently mentioned, "adding this patch was a discussion topic".
But anyway, for the hell of it, I subsequently applied 3/4, and now I at least see
the IPI exception frames:
crash> bt -a
PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff000008dc3f10 PSTATE: 60400149
X29: ffff000008dc3f10 X28: ffff000008dc0000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff000008dc0000 X22: ffff000008bd0270 X21: ffff000008dc0000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 00000000a632f641
X17: 0000ffff7da57880 X16: ffff0000081d9838 X15: 00000000383a0a79
X14: 00000000b2b0b162 X13: 000000005f2cbeec X12: 0000000000045a9e
X11: ffff8000213bd800 X10: 0000000000000850 X9: ffff000008dc0000
X8: 000000010000aa07 X7: 000000000000003d X6: 0015752a00000000
X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
#0 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857bc
#1 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26c8
#2 [ffff000008dc3f80] rest_init at ffff0000087c792c
#3 [ffff000008dc3fa0] start_kernel at ffff000008b10b6c
PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218cff60 PSTATE: 60000349
X29: ffff8000218cff60 X28: ffff8000218cc000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218cc000 X22: ffff000008bd0270 X21: ffff8000218cc000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000005
X14: 0000000c80096000 X13: ffff8000212b6600 X12: 0000000000047ae2
X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218cc000
X8: 000000010000aa07 X7: 0000000000000243 X6: 0015752a00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
#0 [ffff8000218cff60] arch_cpu_idle at ffff0000080857bc
#1 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26c8
#2 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1e8
PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE: 20000000
#0 [user space]
PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
#0 [ffff800020b0ba70] crash_kexec at ffff00000812b0a8
#1 [ffff800020b0ba90] die at ffff000008088ce4
#2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fa8
#3 [ffff800020b0bb00] do_page_fault at ffff000008096810
#4 [ffff800020b0bb60] do_translation_fault at ffff000008096908
#5 [ffff800020b0bb70] do_mem_abort at ffff000008081288
#6 [ffff800020b0bc20] el1_da at ffff000008084564
--- <Exception in kernel> ---
PC: ffff000008457fc8 [sysrq_handle_crash+32]
LR: ffff000008457fb4 [sysrq_handle_crash+12]
SP: ffff800020b0bd40 PSTATE: 60400149
X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
#7 [ffff800020b0bd40] sysrq_handle_crash at ffff000008457fc4
#8 [ffff800020b0bd50] __handle_sysrq at ffff000008458640
#9 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458abc
#10 [ffff800020b0bdb0] proc_reg_write at ffff000008225418
#11 [ffff800020b0bdd0] __vfs_write at ffff0000081c4770
#12 [ffff800020b0be50] vfs_write at ffff0000081c5584
#13 [ffff800020b0be90] sys_write at ffff0000081c68f8
#14 [ffff800020b0bed0] el0_svc_naked at ffff000008084e6c
--- <Exception in user> ---
PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218dbf60 PSTATE: 60000349
X29: ffff8000218dbf60 X28: ffff8000218d8000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218d8000 X22: ffff000008bd0270 X21: ffff8000218d8000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000000
X14: 0000000000000000 X13: 00000000000003e7 X12: 0000000000000016
X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218d8000
X8: ffff8000218c3ab0 X7: 7fffffffffffffff X6: 0015752a00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
#0 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857bc
#1 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26c8
#2 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1e8
PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218dff60 PSTATE: 60000349
X29: ffff8000218dff60 X28: ffff8000218dc000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218dc000 X22: ffff000008bd0270 X21: ffff8000218dc000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000007
X14: 0000000000000000 X13: 000000000000039d X12: 0000000000000000
X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218dc000
X8: 000000010000a479 X7: 7fffffffffffffff X6: 0016694e00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
#0 [ffff8000218dff60] arch_cpu_idle at ffff0000080857bc
#1 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26c8
#2 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1e8
PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218ebf60 PSTATE: 60000349
X29: ffff8000218ebf60 X28: ffff8000218e8000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218e8000 X22: ffff000008bd0270 X21: ffff8000218e8000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 000000000000000a
X14: 0000000000000000 X13: 00000000000003a1 X12: 0000000000000000
X11: ffff8000213a0d00 X10: 0000000000000850 X9: ffff8000218e8000
X8: ffff8000218c53b0 X7: 00000000000001c4 X6: 002887fa00000000
X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
#0 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857bc
#1 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26c8
#2 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1e8
PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
PC: ffff0000080857c0 [arch_cpu_idle+16]
LR: ffff0000080857bc [arch_cpu_idle+12]
SP: ffff8000218eff60 PSTATE: 60400149
X29: ffff8000218eff60 X28: ffff8000218ec000 X27: 0000000000000000
X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
X23: ffff8000218ec000 X22: ffff000008bd0270 X21: ffff8000218ec000
X20: ffff000008dc5b88 X19: 0000000000000000 X18: fffffffffffffffb
X17: 0000ffff9dc8ba48 X16: ffff0000081da048 X15: 003b9aca00000000
X14: 0000000000000000 X13: 0000000000000001 X12: 0000000000000000
X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218ec000
X8: 000000010000a280 X7: ffff8000209b39c8 X6: ffff8000209b39c9
X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
#0 [ffff8000218eff60] arch_cpu_idle at ffff0000080857bc
#1 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26c8
#2 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1e8
crash>
But why do you think that it is an improvement to leave out the transition
to the IRQ stack?
In any case, I haven't even started looking at the "bt -f" part of the patch
because quite frankly, this patchset is so complex that I haven't even
begun to try to understand it. When you said it would be "easier to
maintain", well, perhaps for *you* maybe, but certainly not for me! Without
the kernel's backtrace code from which the current code is based, I don't have
anything to work from anymore. I'm really not sure whether the coverage of the
"corner cases" you referred to make this effort worth it. I would like to see examples
of how the current code fails. But anyway, I will continue to test it to see
if there actually is any significant upgrade from what we already have in place.
>From a kernel debugging perspective, all we really need is a basic unwinder,
and optionally the full dump of the stack data in between those frames. And the
current code does do that at a minimum, and there's much to be said for simplicity.
Dave
>
> For example, taking the 4.6 vmcore that you gave to me, here is the
> current crash utility's output of "bt -a", where the crashing task
> entered crash_kexec() via the sysrq-c page fault exception, and the
> tasks on the other cpus have all entered crash_save_cpu() on their
> IRQ stack as a result of the shutdown IPI, one from user-space and
> the others from the kernel:
>
> crash> bt -a
> PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
> #0 [ffff800022f42e50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022f43010] handle_IPI at ffff00000808e718
> #2 [ffff800022f43040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022f43080] el1_irq at ffff000008084720
> --- <IRQ stack> ---
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff000008dc3f10 PSTATE: 60400149
> X29: ffff000008dc3f10 X28: ffff000008dc0000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff000008dc0000 X22: ffff000008bd0270 X21: ffff000008dc0000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 00000000a632f641
> X17: 0000ffff7da57880 X16: ffff0000081d9838 X15: 00000000383a0a79
> X14: 00000000b2b0b162 X13: 000000005f2cbeec X12: 0000000000045a9e
> X11: ffff8000213bd800 X10: 0000000000000850 X9: ffff000008dc0000
> X8: 000000010000aa07 X7: 000000000000003d X6: 0015752a00000000
> X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> #4 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857c0
> #5 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26cc
> #6 [ffff000008dc3f80] rest_init at ffff0000087c7930
> #7 [ffff000008dc3fa0] start_kernel at ffff000008b10b70
>
> PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
> #0 [ffff800022f56e50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022f57010] handle_IPI at ffff00000808e718
> #2 [ffff800022f57040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022f57080] el1_irq at ffff000008084720
> --- <IRQ stack> ---
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218cff60 PSTATE: 60000349
> X29: ffff8000218cff60 X28: ffff8000218cc000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218cc000 X22: ffff000008bd0270 X21: ffff8000218cc000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000005
> X14: 0000000c80096000 X13: ffff8000212b6600 X12: 0000000000047ae2
> X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218cc000
> X8: 000000010000aa07 X7: 0000000000000243 X6: 0015752a00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> #4 [ffff8000218cff60] arch_cpu_idle at ffff0000080857c0
> #5 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26cc
> #6 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
> #0 [ffff800022f6ae50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022f6b010] handle_IPI at ffff00000808e718
> #2 [ffff800022f6b040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022f6b080] el0_irq_naked at ffff000008084c4c
> --- <IRQ stack> ---
> PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
> X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
> X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
> X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
> X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
> X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
> X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
> X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
> X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
> X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
> X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
> ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE:
> 20000000
>
> PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
> #0 [ffff800020b0ba70] crash_kexec at ffff00000812b0ac
> #1 [ffff800020b0ba90] die at ffff000008088ce8
> #2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fac
> #3 [ffff800020b0bb00] do_page_fault at ffff000008096814
> #4 [ffff800020b0bb60] do_translation_fault at ffff00000809690c
> #5 [ffff800020b0bb70] do_mem_abort at ffff00000808128c
> #6 [ffff800020b0bd40] el1_da at ffff000008084568
> PC: ffff000008457fc8 [sysrq_handle_crash+32]
> LR: ffff000008457fb4 [sysrq_handle_crash+12]
> SP: ffff800020b0bd40 PSTATE: 60400149
> X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
> X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
> X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
> X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
> X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
> X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
> X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
> X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
> X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
> X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
> ORIG_X0: ffff000008dda000 SYSCALLNO: ffff80002104d418
> #7 [ffff800020b0bd50] __handle_sysrq at ffff000008458644
> #8 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458ac0
> #9 [ffff800020b0bdb0] proc_reg_write at ffff00000822541c
> #10 [ffff800020b0bdd0] __vfs_write at ffff0000081c4774
> #11 [ffff800020b0be50] vfs_write at ffff0000081c5588
> #12 [ffff800020b0be90] sys_write at ffff0000081c68fc
> #13 [ffff800020b0bed0] el0_svc_naked at ffff000008084e70
> PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
> X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
> X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
> X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
> X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
> X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
> X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
> X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
> X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
> X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
> X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
> ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
>
> PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
> #0 [ffff800022f92e50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022f93010] handle_IPI at ffff00000808e718
> #2 [ffff800022f93040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022f93080] el1_irq at ffff000008084720
> --- <IRQ stack> ---
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218dbf60 PSTATE: 60000349
> X29: ffff8000218dbf60 X28: ffff8000218d8000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218d8000 X22: ffff000008bd0270 X21: ffff8000218d8000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000000
> X14: 0000000000000000 X13: 00000000000003e7 X12: 0000000000000016
> X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218d8000
> X8: ffff8000218c3ab0 X7: 7fffffffffffffff X6: 0015752a00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> #4 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857c0
> #5 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26cc
> #6 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
> #0 [ffff800022fa6e50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022fa7010] handle_IPI at ffff00000808e718
> #2 [ffff800022fa7040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022fa7080] el1_irq at ffff000008084720
> --- <IRQ stack> ---
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218dff60 PSTATE: 60000349
> X29: ffff8000218dff60 X28: ffff8000218dc000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218dc000 X22: ffff000008bd0270 X21: ffff8000218dc000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000007
> X14: 0000000000000000 X13: 000000000000039d X12: 0000000000000000
> X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218dc000
> X8: 000000010000a479 X7: 7fffffffffffffff X6: 0016694e00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> #4 [ffff8000218dff60] arch_cpu_idle at ffff0000080857c0
> #5 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26cc
> #6 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
> #0 [ffff800022fbae50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022fbb010] handle_IPI at ffff00000808e718
> #2 [ffff800022fbb040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022fbb080] el1_irq at ffff000008084720
> --- <IRQ stack> ---
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218ebf60 PSTATE: 60000349
> X29: ffff8000218ebf60 X28: ffff8000218e8000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218e8000 X22: ffff000008bd0270 X21: ffff8000218e8000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 000000000000000a
> X14: 0000000000000000 X13: 00000000000003a1 X12: 0000000000000000
> X11: ffff8000213a0d00 X10: 0000000000000850 X9: ffff8000218e8000
> X8: ffff8000218c53b0 X7: 00000000000001c4 X6: 002887fa00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> #4 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857c0
> #5 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26cc
> #6 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
> #0 [ffff800022fcee50] crash_save_cpu at ffff00000812ae44
> #1 [ffff800022fcf010] handle_IPI at ffff00000808e718
> #2 [ffff800022fcf040] gic_handle_irq at ffff0000080815f8
> #3 [ffff800022fcf080] el1_irq at ffff000008084720
> --- <IRQ stack> ---
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218eff60 PSTATE: 60400149
> X29: ffff8000218eff60 X28: ffff8000218ec000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218ec000 X22: ffff000008bd0270 X21: ffff8000218ec000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: fffffffffffffffb
> X17: 0000ffff9dc8ba48 X16: ffff0000081da048 X15: 003b9aca00000000
> X14: 0000000000000000 X13: 0000000000000001 X12: 0000000000000000
> X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218ec000
> X8: 000000010000a280 X7: ffff8000209b39c8 X6: ffff8000209b39c9
> X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> #4 [ffff8000218eff60] arch_cpu_idle at ffff0000080857c0
> #5 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26cc
> #6 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1ec
> crash>
>
> And that's precisely what we want to see. (other than perhaps the
> in-kernel exception frame's dump of the ORIG_X0 and SYSCALL registers)
>
> Now, with v5's patches 1/2 and 2/2 patches applied, this is what gets
> displayed:
>
> crash> bt -a
> PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
> #0 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857c0
> #1 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26cc
> #2 [ffff000008dc3f80] rest_init at ffff0000087c7930
> #3 [ffff000008dc3fa0] start_kernel at ffff000008b10b70
>
> PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
> #0 [ffff8000218cff60] arch_cpu_idle at ffff0000080857c0
> #1 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26cc
> #2 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
> PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
> X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
> X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
> X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
> X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
> X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
> X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
> X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
> X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
> X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
> X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
> ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE:
> 20000000
>
> #0 [user space]
>
> PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
> #0 [ffff800020b0ba70] crash_kexec at ffff00000812b0ac
> #1 [ffff800020b0ba90] die at ffff000008088ce8
> #2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fac
> #3 [ffff800020b0bb00] do_page_fault at ffff000008096814
> #4 [ffff800020b0bb60] do_translation_fault at ffff00000809690c
> #5 [ffff800020b0bb70] do_mem_abort at ffff00000808128c
> #6 [ffff800020b0bc20] el1_da at ffff000008084568
> --- <Exception in kernel> ---
> PC: ffff000008457fc8 [sysrq_handle_crash+32]
> LR: ffff000008457fb4 [sysrq_handle_crash+12]
> SP: ffff800020b0bd40 PSTATE: 60400149
> X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
> X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
> X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
> X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
> X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
> X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
> X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
> X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
> X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
> X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
>
> #7 [ffff800020b0bd40] sysrq_handle_crash at ffff000008457fc8
> #8 [ffff800020b0bd50] __handle_sysrq at ffff000008458644
> #9 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458ac0
> #10 [ffff800020b0bdb0] proc_reg_write at ffff00000822541c
> #11 [ffff800020b0bdd0] __vfs_write at ffff0000081c4774
> #12 [ffff800020b0be50] vfs_write at ffff0000081c5588
> #13 [ffff800020b0be90] sys_write at ffff0000081c68fc
> #14 [ffff800020b0bed0] el0_svc_naked at ffff000008084e70
> --- <Exception in user> ---
> PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
> X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
> X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
> X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
> X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
> X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
> X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
> X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
> X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
> X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
> X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
> ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
>
> PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
> #0 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857c0
> #1 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26cc
> #2 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
> #0 [ffff8000218dff60] arch_cpu_idle at ffff0000080857c0
> #1 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26cc
> #2 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
> #0 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857c0
> #1 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26cc
> #2 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1ec
>
> PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
> #0 [ffff8000218eff60] arch_cpu_idle at ffff0000080857c0
> #1 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26cc
> #2 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1ec
> crash>
>
> What happened? This can't possibly be what you intended, is it?
>
> Dave
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
More information about the Crash-utility
mailing list