[Crash-utility] [PATCH v5 0/4] arm64: more improvement of bt -f
AKASHI Takahiro
takahiro.akashi at linaro.org
Thu Jun 30 00:16:38 UTC 2016
On Wed, Jun 29, 2016 at 04:44:41PM -0400, Dave Anderson wrote:
>
> > Hi Takahiro,
> >
> > I applied patches 1/2 and 2/2 from the v5 patchset. But I can't
> > believe the results are what you intended?
>
> Obviously I meant 1/4 and 2/4 above.
>
> However, I was under the impression that the 3/4 patch was a standalone
> patch that only served to change the text address displayed,
Right. I might have made some mistake when I squashed up the changes.
> and that
> "adding this patch was a discussion topic": ???
>
> > On arm64, the link register (LR) holds a return address, which is the one
> > just after a branch instruction. So using a saved lr as PC for backtracing
> > might cause some confusion.
> > For example, in kernel/entry.S,
> > work_resched:
> > ...
> > bl schedule
> >
> > ret_to_user:
> > ...
> >
> > The current code shows "ret_o_user", instead of "work_resched",
> > as a caller of schedule().
> >
> > This patch corrects a PC by decrementing it by 4.
> > But please note that this change may also make people a bit confused
> > because a value of LR in the stack dump of "bt -f" doesn't match with
> > an address in one-line summary.
> >
> > #2 [ffffcc7511407eb0] schedule at ffff0000d628aee0
> > ffffcc7511407eb0: ffffcc6d22f23080 ffff0000d5b44d6c <= LR
> > ffffcc7511407ec0: ffffcc7511407ed0 0000000000000000
> > #3 [ffffcc7511407ed0] work_resched at ffff0000d5b44d68 <= correcrted PC
> >
> > Signed-off-by: AKASHI Takahiro <takahiro akashi linaro org>
>
> ...and as you subsequently mentioned, "adding this patch was a discussion topic".
>
> But anyway, for the hell of it, I subsequently applied 3/4, and now I at least see
> the IPI exception frames:
Sure that I did make this change recently.
I mean, it is intentional but it should not have gone with patch#3 but patch#1.
>
> crash> bt -a
> PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff000008dc3f10 PSTATE: 60400149
> X29: ffff000008dc3f10 X28: ffff000008dc0000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff000008dc0000 X22: ffff000008bd0270 X21: ffff000008dc0000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 00000000a632f641
> X17: 0000ffff7da57880 X16: ffff0000081d9838 X15: 00000000383a0a79
> X14: 00000000b2b0b162 X13: 000000005f2cbeec X12: 0000000000045a9e
> X11: ffff8000213bd800 X10: 0000000000000850 X9: ffff000008dc0000
> X8: 000000010000aa07 X7: 000000000000003d X6: 0015752a00000000
> X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
>
> #0 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857bc
> #1 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26c8
> #2 [ffff000008dc3f80] rest_init at ffff0000087c792c
> #3 [ffff000008dc3fa0] start_kernel at ffff000008b10b6c
>
> PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218cff60 PSTATE: 60000349
> X29: ffff8000218cff60 X28: ffff8000218cc000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218cc000 X22: ffff000008bd0270 X21: ffff8000218cc000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000005
> X14: 0000000c80096000 X13: ffff8000212b6600 X12: 0000000000047ae2
> X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218cc000
> X8: 000000010000aa07 X7: 0000000000000243 X6: 0015752a00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
>
> #0 [ffff8000218cff60] arch_cpu_idle at ffff0000080857bc
> #1 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26c8
> #2 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1e8
>
> PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
> PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
> X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
> X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
> X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
> X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
> X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
> X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
> X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
> X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
> X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
> X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
> ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE: 20000000
>
> #0 [user space]
>
> PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
> #0 [ffff800020b0ba70] crash_kexec at ffff00000812b0a8
> #1 [ffff800020b0ba90] die at ffff000008088ce4
> #2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fa8
> #3 [ffff800020b0bb00] do_page_fault at ffff000008096810
> #4 [ffff800020b0bb60] do_translation_fault at ffff000008096908
> #5 [ffff800020b0bb70] do_mem_abort at ffff000008081288
> #6 [ffff800020b0bc20] el1_da at ffff000008084564
> --- <Exception in kernel> ---
> PC: ffff000008457fc8 [sysrq_handle_crash+32]
> LR: ffff000008457fb4 [sysrq_handle_crash+12]
> SP: ffff800020b0bd40 PSTATE: 60400149
> X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
> X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
> X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
> X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
> X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
> X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
> X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
> X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
> X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
> X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
>
> #7 [ffff800020b0bd40] sysrq_handle_crash at ffff000008457fc4
> #8 [ffff800020b0bd50] __handle_sysrq at ffff000008458640
> #9 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458abc
> #10 [ffff800020b0bdb0] proc_reg_write at ffff000008225418
> #11 [ffff800020b0bdd0] __vfs_write at ffff0000081c4770
> #12 [ffff800020b0be50] vfs_write at ffff0000081c5584
> #13 [ffff800020b0be90] sys_write at ffff0000081c68f8
> #14 [ffff800020b0bed0] el0_svc_naked at ffff000008084e6c
> --- <Exception in user> ---
> PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
> X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
> X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
> X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
> X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
> X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
> X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
> X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
> X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
> X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
> X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
> ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
>
> PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218dbf60 PSTATE: 60000349
> X29: ffff8000218dbf60 X28: ffff8000218d8000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218d8000 X22: ffff000008bd0270 X21: ffff8000218d8000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000000
> X14: 0000000000000000 X13: 00000000000003e7 X12: 0000000000000016
> X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218d8000
> X8: ffff8000218c3ab0 X7: 7fffffffffffffff X6: 0015752a00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
>
> #0 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857bc
> #1 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26c8
> #2 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1e8
>
> PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218dff60 PSTATE: 60000349
> X29: ffff8000218dff60 X28: ffff8000218dc000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218dc000 X22: ffff000008bd0270 X21: ffff8000218dc000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000007
> X14: 0000000000000000 X13: 000000000000039d X12: 0000000000000000
> X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218dc000
> X8: 000000010000a479 X7: 7fffffffffffffff X6: 0016694e00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
>
> #0 [ffff8000218dff60] arch_cpu_idle at ffff0000080857bc
> #1 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26c8
> #2 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1e8
>
> PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218ebf60 PSTATE: 60000349
> X29: ffff8000218ebf60 X28: ffff8000218e8000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218e8000 X22: ffff000008bd0270 X21: ffff8000218e8000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 000000000000000a
> X14: 0000000000000000 X13: 00000000000003a1 X12: 0000000000000000
> X11: ffff8000213a0d00 X10: 0000000000000850 X9: ffff8000218e8000
> X8: ffff8000218c53b0 X7: 00000000000001c4 X6: 002887fa00000000
> X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
>
> #0 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857bc
> #1 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26c8
> #2 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1e8
>
> PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
> PC: ffff0000080857c0 [arch_cpu_idle+16]
> LR: ffff0000080857bc [arch_cpu_idle+12]
> SP: ffff8000218eff60 PSTATE: 60400149
> X29: ffff8000218eff60 X28: ffff8000218ec000 X27: 0000000000000000
> X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> X23: ffff8000218ec000 X22: ffff000008bd0270 X21: ffff8000218ec000
> X20: ffff000008dc5b88 X19: 0000000000000000 X18: fffffffffffffffb
> X17: 0000ffff9dc8ba48 X16: ffff0000081da048 X15: 003b9aca00000000
> X14: 0000000000000000 X13: 0000000000000001 X12: 0000000000000000
> X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218ec000
> X8: 000000010000a280 X7: ffff8000209b39c8 X6: ffff8000209b39c9
> X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
> X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
>
> #0 [ffff8000218eff60] arch_cpu_idle at ffff0000080857bc
> #1 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26c8
> #2 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1e8
> crash>
>
> But why do you think that it is an improvement to leave out the transition
> to the IRQ stack?
I explained the reason in my previous e-mail.
I still believe that it is an improvement.
> In any case, I haven't even started looking at the "bt -f" part of the patch
> because quite frankly, this patchset is so complex that I haven't even
> begun to try to understand it. When you said it would be "easier to
> maintain", well, perhaps for *you* maybe, but certainly not for me!
Do you think so?
It is because you are the author.
The changes I made were big and look complex to you,
but the resulting code is simple enough for others IMO.
I tried to make the main loop of arm64_back_trace_cmd() simple and
quite resemble the counterpart of dump_backtrace() in the kernel.
The differences come mainly from the facts:
1.you (crash util) need to display not only a one-line frame summary
(with a value of stack/frame pointer), but also a full dump of stack
2.your current code will miss a *very important* stack frame that is
the one exactly when an interrupt takes place.
I believe that the latter is a big improvement.
Yeah, due to the changes, arm64_unwind_frame() may get a bit complicated,
but it is very naive implementation based on the nature of the kernel's
stack usage (or PCS for ARM), the complexity is not the result of my poor
skill of coding.
In other words, the complexity is now *encapsulated* in that function.
In addition, I wrote down that function step-by-step, from generic cases
to exceptional cases, with a bunch of comments for better understandings.
And you will be able to remove such exceptional cases *if* you don't like
them.
Given those stuffs, I think that my code is easier to maintain.
Again, surely up to you.
-Takahiro AKASHI
> Without
> the kernel's backtrace code from which the current code is based, I don't have
> anything to work from anymore. I'm really not sure whether the coverage of the
> "corner cases" you referred to make this effort worth it. I would like to see examples
> of how the current code fails. But anyway, I will continue to test it to see
> if there actually is any significant upgrade from what we already have in place.
> >From a kernel debugging perspective, all we really need is a basic unwinder,
> and optionally the full dump of the stack data in between those frames. And the
> current code does do that at a minimum, and there's much to be said for simplicity.
>
> Dave
>
>
>
>
>
>
>
>
>
> >
> > For example, taking the 4.6 vmcore that you gave to me, here is the
> > current crash utility's output of "bt -a", where the crashing task
> > entered crash_kexec() via the sysrq-c page fault exception, and the
> > tasks on the other cpus have all entered crash_save_cpu() on their
> > IRQ stack as a result of the shutdown IPI, one from user-space and
> > the others from the kernel:
> >
> > crash> bt -a
> > PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
> > #0 [ffff800022f42e50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022f43010] handle_IPI at ffff00000808e718
> > #2 [ffff800022f43040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022f43080] el1_irq at ffff000008084720
> > --- <IRQ stack> ---
> > PC: ffff0000080857c0 [arch_cpu_idle+16]
> > LR: ffff0000080857bc [arch_cpu_idle+12]
> > SP: ffff000008dc3f10 PSTATE: 60400149
> > X29: ffff000008dc3f10 X28: ffff000008dc0000 X27: 0000000000000000
> > X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> > X23: ffff000008dc0000 X22: ffff000008bd0270 X21: ffff000008dc0000
> > X20: ffff000008dc5b88 X19: 0000000000000000 X18: 00000000a632f641
> > X17: 0000ffff7da57880 X16: ffff0000081d9838 X15: 00000000383a0a79
> > X14: 00000000b2b0b162 X13: 000000005f2cbeec X12: 0000000000045a9e
> > X11: ffff8000213bd800 X10: 0000000000000850 X9: ffff000008dc0000
> > X8: 000000010000aa07 X7: 000000000000003d X6: 0015752a00000000
> > X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
> > X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> > ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> > #4 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857c0
> > #5 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26cc
> > #6 [ffff000008dc3f80] rest_init at ffff0000087c7930
> > #7 [ffff000008dc3fa0] start_kernel at ffff000008b10b70
> >
> > PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
> > #0 [ffff800022f56e50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022f57010] handle_IPI at ffff00000808e718
> > #2 [ffff800022f57040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022f57080] el1_irq at ffff000008084720
> > --- <IRQ stack> ---
> > PC: ffff0000080857c0 [arch_cpu_idle+16]
> > LR: ffff0000080857bc [arch_cpu_idle+12]
> > SP: ffff8000218cff60 PSTATE: 60000349
> > X29: ffff8000218cff60 X28: ffff8000218cc000 X27: 0000000000000000
> > X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> > X23: ffff8000218cc000 X22: ffff000008bd0270 X21: ffff8000218cc000
> > X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> > X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000005
> > X14: 0000000c80096000 X13: ffff8000212b6600 X12: 0000000000047ae2
> > X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218cc000
> > X8: 000000010000aa07 X7: 0000000000000243 X6: 0015752a00000000
> > X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> > X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> > ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> > #4 [ffff8000218cff60] arch_cpu_idle at ffff0000080857c0
> > #5 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26cc
> > #6 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
> > #0 [ffff800022f6ae50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022f6b010] handle_IPI at ffff00000808e718
> > #2 [ffff800022f6b040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022f6b080] el0_irq_naked at ffff000008084c4c
> > --- <IRQ stack> ---
> > PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
> > X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
> > X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
> > X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
> > X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
> > X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
> > X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
> > X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
> > X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
> > X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
> > X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
> > ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE:
> > 20000000
> >
> > PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
> > #0 [ffff800020b0ba70] crash_kexec at ffff00000812b0ac
> > #1 [ffff800020b0ba90] die at ffff000008088ce8
> > #2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fac
> > #3 [ffff800020b0bb00] do_page_fault at ffff000008096814
> > #4 [ffff800020b0bb60] do_translation_fault at ffff00000809690c
> > #5 [ffff800020b0bb70] do_mem_abort at ffff00000808128c
> > #6 [ffff800020b0bd40] el1_da at ffff000008084568
> > PC: ffff000008457fc8 [sysrq_handle_crash+32]
> > LR: ffff000008457fb4 [sysrq_handle_crash+12]
> > SP: ffff800020b0bd40 PSTATE: 60400149
> > X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
> > X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
> > X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
> > X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
> > X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
> > X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
> > X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
> > X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
> > X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
> > X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
> > ORIG_X0: ffff000008dda000 SYSCALLNO: ffff80002104d418
> > #7 [ffff800020b0bd50] __handle_sysrq at ffff000008458644
> > #8 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458ac0
> > #9 [ffff800020b0bdb0] proc_reg_write at ffff00000822541c
> > #10 [ffff800020b0bdd0] __vfs_write at ffff0000081c4774
> > #11 [ffff800020b0be50] vfs_write at ffff0000081c5588
> > #12 [ffff800020b0be90] sys_write at ffff0000081c68fc
> > #13 [ffff800020b0bed0] el0_svc_naked at ffff000008084e70
> > PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
> > X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
> > X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
> > X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
> > X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
> > X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
> > X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
> > X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
> > X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
> > X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
> > X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
> > ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
> >
> > PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
> > #0 [ffff800022f92e50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022f93010] handle_IPI at ffff00000808e718
> > #2 [ffff800022f93040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022f93080] el1_irq at ffff000008084720
> > --- <IRQ stack> ---
> > PC: ffff0000080857c0 [arch_cpu_idle+16]
> > LR: ffff0000080857bc [arch_cpu_idle+12]
> > SP: ffff8000218dbf60 PSTATE: 60000349
> > X29: ffff8000218dbf60 X28: ffff8000218d8000 X27: 0000000000000000
> > X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> > X23: ffff8000218d8000 X22: ffff000008bd0270 X21: ffff8000218d8000
> > X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> > X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000000
> > X14: 0000000000000000 X13: 00000000000003e7 X12: 0000000000000016
> > X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218d8000
> > X8: ffff8000218c3ab0 X7: 7fffffffffffffff X6: 0015752a00000000
> > X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> > X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> > ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> > #4 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857c0
> > #5 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26cc
> > #6 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
> > #0 [ffff800022fa6e50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022fa7010] handle_IPI at ffff00000808e718
> > #2 [ffff800022fa7040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022fa7080] el1_irq at ffff000008084720
> > --- <IRQ stack> ---
> > PC: ffff0000080857c0 [arch_cpu_idle+16]
> > LR: ffff0000080857bc [arch_cpu_idle+12]
> > SP: ffff8000218dff60 PSTATE: 60000349
> > X29: ffff8000218dff60 X28: ffff8000218dc000 X27: 0000000000000000
> > X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> > X23: ffff8000218dc000 X22: ffff000008bd0270 X21: ffff8000218dc000
> > X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> > X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 0000000000000007
> > X14: 0000000000000000 X13: 000000000000039d X12: 0000000000000000
> > X11: ffff0000087ef470 X10: 0000000000000850 X9: ffff8000218dc000
> > X8: 000000010000a479 X7: 7fffffffffffffff X6: 0016694e00000000
> > X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> > X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> > ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> > #4 [ffff8000218dff60] arch_cpu_idle at ffff0000080857c0
> > #5 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26cc
> > #6 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
> > #0 [ffff800022fbae50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022fbb010] handle_IPI at ffff00000808e718
> > #2 [ffff800022fbb040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022fbb080] el1_irq at ffff000008084720
> > --- <IRQ stack> ---
> > PC: ffff0000080857c0 [arch_cpu_idle+16]
> > LR: ffff0000080857bc [arch_cpu_idle+12]
> > SP: ffff8000218ebf60 PSTATE: 60000349
> > X29: ffff8000218ebf60 X28: ffff8000218e8000 X27: 0000000000000000
> > X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> > X23: ffff8000218e8000 X22: ffff000008bd0270 X21: ffff8000218e8000
> > X20: ffff000008dc5b88 X19: 0000000000000000 X18: 000000000000016d
> > X17: 0000007f8b122780 X16: ffffffc0001adf68 X15: 000000000000000a
> > X14: 0000000000000000 X13: 00000000000003a1 X12: 0000000000000000
> > X11: ffff8000213a0d00 X10: 0000000000000850 X9: ffff8000218e8000
> > X8: ffff8000218c53b0 X7: 00000000000001c4 X6: 002887fa00000000
> > X5: 0100000000000000 X4: 00000000000003c0 X3: 0000000000000000
> > X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> > ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> > #4 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857c0
> > #5 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26cc
> > #6 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
> > #0 [ffff800022fcee50] crash_save_cpu at ffff00000812ae44
> > #1 [ffff800022fcf010] handle_IPI at ffff00000808e718
> > #2 [ffff800022fcf040] gic_handle_irq at ffff0000080815f8
> > #3 [ffff800022fcf080] el1_irq at ffff000008084720
> > --- <IRQ stack> ---
> > PC: ffff0000080857c0 [arch_cpu_idle+16]
> > LR: ffff0000080857bc [arch_cpu_idle+12]
> > SP: ffff8000218eff60 PSTATE: 60400149
> > X29: ffff8000218eff60 X28: ffff8000218ec000 X27: 0000000000000000
> > X26: ffff000008dc5000 X25: ffff000008dc5c1c X24: ffff000008dc5000
> > X23: ffff8000218ec000 X22: ffff000008bd0270 X21: ffff8000218ec000
> > X20: ffff000008dc5b88 X19: 0000000000000000 X18: fffffffffffffffb
> > X17: 0000ffff9dc8ba48 X16: ffff0000081da048 X15: 003b9aca00000000
> > X14: 0000000000000000 X13: 0000000000000001 X12: 0000000000000000
> > X11: ffff800021089980 X10: 0000000000000850 X9: ffff8000218ec000
> > X8: 000000010000a280 X7: ffff8000209b39c8 X6: ffff8000209b39c9
> > X5: 0100000000000000 X4: 00000000000001c0 X3: 0000000000000000
> > X2: 0000000000000001 X1: 0000000000000000 X0: 0000000000000000
> > ORIG_X0: 0000000000000000 SYSCALLNO: 7fffffffffffffff
> > #4 [ffff8000218eff60] arch_cpu_idle at ffff0000080857c0
> > #5 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26cc
> > #6 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1ec
> > crash>
> >
> > And that's precisely what we want to see. (other than perhaps the
> > in-kernel exception frame's dump of the ORIG_X0 and SYSCALL registers)
> >
> > Now, with v5's patches 1/2 and 2/2 patches applied, this is what gets
> > displayed:
> >
> > crash> bt -a
> > PID: 0 TASK: ffff000008dcd900 CPU: 0 COMMAND: "swapper/0"
> > #0 [ffff000008dc3f10] arch_cpu_idle at ffff0000080857c0
> > #1 [ffff000008dc3f20] cpu_startup_entry at ffff0000080f26cc
> > #2 [ffff000008dc3f80] rest_init at ffff0000087c7930
> > #3 [ffff000008dc3fa0] start_kernel at ffff000008b10b70
> >
> > PID: 0 TASK: ffff8000218c0c80 CPU: 1 COMMAND: "swapper/1"
> > #0 [ffff8000218cff60] arch_cpu_idle at ffff0000080857c0
> > #1 [ffff8000218cff70] cpu_startup_entry at ffff0000080f26cc
> > #2 [ffff8000218cffd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 1324 TASK: ffff80002018be80 CPU: 2 COMMAND: "dhry"
> > PC: 00000000004016a4 LR: 00000000004016a4 SP: 0000ffffc10c40a0
> > X29: 0000ffffc10c40a0 X28: 0000000000000000 X27: 0000000000000000
> > X26: 0000000000000000 X25: 0000000000402138 X24: 00000000004021f0
> > X23: 0000000000000000 X22: 0000000000000000 X21: 00000000004001a0
> > X20: 0000000000000000 X19: 0000000000000000 X18: 0000000000000000
> > X17: 0000000000000001 X16: 0000000000000000 X15: 0000000000493000
> > X14: 0000000000498000 X13: ffffffffffffffff X12: 0000000000000005
> > X11: 000000000000001e X10: 0101010101010101 X9: fffffffff59a9190
> > X8: 7f7f7f7f7f7f7f7f X7: 1f535226301f2b4c X6: 00000003001d1000
> > X5: 00101d0003000000 X4: 0000000000000000 X3: 4952545320454d4f
> > X2: 0000000010c35b40 X1: 0000000000000011 X0: 0000000010c35b40
> > ORIG_X0: 0000000000498700 SYSCALLNO: ffffffffffffffff PSTATE:
> > 20000000
> >
> > #0 [user space]
> >
> > PID: 1223 TASK: ffff800020ef5780 CPU: 3 COMMAND: "sh"
> > #0 [ffff800020b0ba70] crash_kexec at ffff00000812b0ac
> > #1 [ffff800020b0ba90] die at ffff000008088ce8
> > #2 [ffff800020b0bad0] __do_kernel_fault at ffff000008098fac
> > #3 [ffff800020b0bb00] do_page_fault at ffff000008096814
> > #4 [ffff800020b0bb60] do_translation_fault at ffff00000809690c
> > #5 [ffff800020b0bb70] do_mem_abort at ffff00000808128c
> > #6 [ffff800020b0bc20] el1_da at ffff000008084568
> > --- <Exception in kernel> ---
> > PC: ffff000008457fc8 [sysrq_handle_crash+32]
> > LR: ffff000008457fb4 [sysrq_handle_crash+12]
> > SP: ffff800020b0bd40 PSTATE: 60400149
> > X29: ffff800020b0bd40 X28: ffff800020b08000 X27: ffff0000087e2000
> > X26: 0000000000000040 X25: 0000000000000120 X24: 0000000000000015
> > X23: 0000000000000000 X22: 0000000000000009 X21: ffff000008e071b0
> > X20: 0000000000000063 X19: ffff000008dda000 X18: 0000000000000006
> > X17: 0000ffff83d266c0 X16: ffff0000081c68b8 X15: ffff000008e6cc95
> > X14: 0000ffff83cc28f0 X13: ffff000008e6c758 X12: ffff000008dda7a0
> > X11: 0000000000000106 X10: 0000000000000002 X9: 0000000000000001
> > X8: 0000000000000106 X7: 0000000000000000 X6: ffff800022f337a4
> > X5: 00000000000047d4 X4: 0000000000000000 X3: 0000000000000000
> > X2: ffff000008dda7b8 X1: 0000000000000000 X0: 0000000000000001
> >
> > #7 [ffff800020b0bd40] sysrq_handle_crash at ffff000008457fc8
> > #8 [ffff800020b0bd50] __handle_sysrq at ffff000008458644
> > #9 [ffff800020b0bd90] write_sysrq_trigger at ffff000008458ac0
> > #10 [ffff800020b0bdb0] proc_reg_write at ffff00000822541c
> > #11 [ffff800020b0bdd0] __vfs_write at ffff0000081c4774
> > #12 [ffff800020b0be50] vfs_write at ffff0000081c5588
> > #13 [ffff800020b0be90] sys_write at ffff0000081c68fc
> > #14 [ffff800020b0bed0] el0_svc_naked at ffff000008084e70
> > --- <Exception in user> ---
> > PC: 0000ffff83d7b1e8 LR: 0000ffff83d2d700 SP: 0000fffffe197460
> > X29: 0000fffffe197460 X28: 0000000000000000 X27: 0000000000000000
> > X26: 000000001e9e40c8 X25: 00000000004c9be8 X24: 00000000004ec000
> > X23: 0000000000000001 X22: 0000000000000002 X21: 0000ffff83e02500
> > X20: 000000001e9da808 X19: 0000000000000002 X18: 0000000000000015
> > X17: 0000ffff83d266c0 X16: 0000000000000000 X15: 0015a60d0a000000
> > X14: 0000ffff83cc28f0 X13: 0000000000000018 X12: 0000000000000008
> > X11: 0101010101010101 X10: 7f7f7f7f7f7f7f7f X9: fefefefeff00ff62
> > X8: 0000000000000040 X7: fefefefefefefefe X6: 000000001e9da808
> > X5: 0000ffff83d055c8 X4: 0000000000000000 X3: 0000000000000000
> > X2: 0000000000000002 X1: 000000001e9da808 X0: 0000000000000001
> > ORIG_X0: 0000000000000001 SYSCALLNO: 40 PSTATE: 20000000
> >
> > PID: 0 TASK: ffff8000218c3200 CPU: 4 COMMAND: "swapper/4"
> > #0 [ffff8000218dbf60] arch_cpu_idle at ffff0000080857c0
> > #1 [ffff8000218dbf70] cpu_startup_entry at ffff0000080f26cc
> > #2 [ffff8000218dbfd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 0 TASK: ffff8000218c3e80 CPU: 5 COMMAND: "swapper/5"
> > #0 [ffff8000218dff60] arch_cpu_idle at ffff0000080857c0
> > #1 [ffff8000218dff70] cpu_startup_entry at ffff0000080f26cc
> > #2 [ffff8000218dffd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 0 TASK: ffff8000218c4b00 CPU: 6 COMMAND: "swapper/6"
> > #0 [ffff8000218ebf60] arch_cpu_idle at ffff0000080857c0
> > #1 [ffff8000218ebf70] cpu_startup_entry at ffff0000080f26cc
> > #2 [ffff8000218ebfd0] secondary_start_kernel at ffff00000808e1ec
> >
> > PID: 0 TASK: ffff8000218c5780 CPU: 7 COMMAND: "swapper/7"
> > #0 [ffff8000218eff60] arch_cpu_idle at ffff0000080857c0
> > #1 [ffff8000218eff70] cpu_startup_entry at ffff0000080f26cc
> > #2 [ffff8000218effd0] secondary_start_kernel at ffff00000808e1ec
> > crash>
> >
> > What happened? This can't possibly be what you intended, is it?
> >
> > Dave
> >
> > --
> > Crash-utility mailing list
> > Crash-utility at redhat.com
> > https://www.redhat.com/mailman/listinfo/crash-utility
> >
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
More information about the Crash-utility
mailing list