[Crash-utility] [PATCH] ppc64: fix 'bt' command for vmcore captured with fadump.
Hari Bathini
hbathini at linux.vnet.ibm.com
Tue Jan 24 17:43:48 UTC 2017
On Monday 23 January 2017 11:43 PM, Dave Anderson wrote:
>
> ----- Original Message -----
>>
>> On Saturday 21 January 2017 02:00 AM, Dave Anderson wrote:
>>> ----- Original Message -----
>>>
>>> ... [cut] ...
>>>
>>>>> Also, the exception frame doesn't even show the [bracketed] type of
>>>>> exception
>>>>> that occurred -- it's just a register dump followed by the remainder of
>>>>> the
>>>>> backtrace. Upon a quick glance, it's not obvious that they are even
>>>>> active
>>>>> tasks. And traditionally, all of the other architectures have always
>>>>> dumped
>>>>> a full trace.
>>>>>
>>>>> I'm not sure what the mechanism is for shutting down the non-active
>>>>> FADUMP tasks, so that's why I asked if you could restrict this change
>>>>> to just those types of dumps. (For that matter, is it even possible to
>>>>> differentiate a real kdump from an FADUMP dumpfile -- aside from a
>>>> Hi Dave,
>>>>
>>>> Differentiating a kdump and fadump dumpfile is not possible except that
>>>> the
>>>> stack search would invariably fail and ptregs are guaranteed to be saved
>>>> by
>>>> firmware in case of fadump. Posted v2 that doesn't change bt output for
>>>> anything
>>>> but active tasks in case of fadump..
>>> Ok, so let me get this straight. The only difference I see with the v2
>>> patch
>>> is that fadump non-panicking active tasks change from something like this:
>>>
>>> PID: 0 TASK: c000000000e7f6d0 CPU: 0 COMMAND: "swapper"
>>> #0 [c000000000f2ba30] (null) at 3aae291c67 (unreliable)
>>> #1 [c000000000f2bae0] .tick_dev_program_event at c0000000000d16fc
>>> #2 [c000000000f2bb90] .__hrtimer_start_range_ns at c0000000000c4bcc
>>> #3 [c000000000f2bcb0] .tick_nohz_stop_sched_tick at c0000000000d2d30
>>> #4 [c000000000f2bdc0] .cpu_idle at c000000000015bf0
>>> #5 [c000000000f2be70] .rest_init at c000000000009de4
>>> #6 [c000000000f2bef0] .start_kernel at c000000000850eb4
>>> #7 [c000000000f2bf90] .start_here_common at c0000000000083d8
>>>
>>> to this:
>>>
>>> PID: 0 TASK: c000000000e7f6d0 CPU: 0 COMMAND: "swapper"
>>> #0 [c000000000f2bd50] (null) at 0 (unreliable)
>>> #1 [c000000000f2bdc0] .cpu_idle at c000000000015bf0
>>> #2 [c000000000f2be70] .rest_init at c000000000009de4
>>> #3 [c000000000f2bef0] .start_kernel at c000000000850eb4
>>> #4 [c000000000f2bf90] .start_here_common at c0000000000083d8
>>>
>>> But with your v1 patch, you also dumped the exception frame:
>>>
>>> PID: 0 TASK: c000000000e7f6d0 CPU: 0 COMMAND: "swapper"
>>> R0: 0000000000000000 R1: c000000000f2bd50 R2: c000000000f27628
>>> R3: 0000000000000000 R4: 0000000000000000 R5: 8000000002144400
>>> R6: 800000001314c4f8 R7: 0000000000000000 R8: 0000000000000000
>>> R9: ffffffffffffffff R10: 0000000000000000 R11: 80003fbff901700c
>>> R12: 0000000000000000 R13: c000000000ff2500 R14: 0000000001a3fa58
>>> R15: 00000000002230a8 R16: 0000000000223150 R17: 0000000000223144
>>> R18: 0000000000c8a098 R19: 0000000002b13a58 R20: 0000000000000000
>>> R21: 0000000002b135d8 R22: 0000000002b13530 R23: 0000000002280000
>>> R24: 0000000002b135f0 R25: c000000000fd5c48 R26: c0000000010942f0
>>> R27: c0000000010942f0 R28: c0000000005fd168 R29: 0000000000000008
>>> R30: c000000000eb1d68 R31: c000000000f28080
>>> NIP: c000000000055730 MSR: 8000000000009032 OR3: 0000000000000000
>>> CTR: 0000000000000000 LR: c000000000057350 XER: 0000000000000000
>>> CCR: 0000000024000048 MQ: 0000000000000000 DAR: 000001000ad763b0
>>> DSISR: 0000000000000000 Syscall Result: 0000000000000000
>>> NIP [c000000000055730] .plpar_hcall_norets
>>> LR [c000000000057350] .pseries_shared_idle_sleep
>>> #0 [c000000000f2bd50] (null) at 0 (unreliable)
>>> #1 [c000000000f2bdc0] .cpu_idle at c000000000015bf0
>>> #2 [c000000000f2be70] .rest_init at c000000000009de4
>>> #3 [c000000000f2bef0] .start_kernel at c000000000850eb4
>>> #4 [c000000000f2bf90] .start_here_common at c0000000000083d8
>>>
>>> Again, I don't understand how the non-panicking active tasks are stopped
>>> by the fadump facility, but is it because you cannot differentiate kdumps
>>> from fadumps that you don't show the exception frame with the v2 patch?
>> Hi Dave,
>>
>> The crashing cpu makes rtas call ibm,os-term to the firmware which
>> saves the regs info of all online cpus. AFAIK, there is no exception frame
>> marker (which we are using to detect one) set for this stack frames
>> by the kernel. With v1, I was printing the registers without looking for
>> exception frame marker, if the registers are saved..
>>
>>> Would it be possible to also show the exception frame type in brackets and
>>> the register dump for those fadump non-panicking active tasks?
>>>
>> Hmmm.. Let me have a hard look at this.
>> Will try and improve this..
> Hari,
>
> I was tinkering around with ppc64_get_dumpfile_stack_frame() from your v2 patch,
> and this seems to work:
>
> else {
> *ksp = pt_regs->gpr[1];
> if (IS_KVADDR(*ksp)) {
> readmem(*ksp+16, KVADDR, nip, sizeof(ulong),
> "Regs NIP value", FAULT_ON_ERROR);
> + ppc64_print_regs(pt_regs);
> return TRUE;
> } else {
> if (IN_TASK_VMA(bt_in->task, *ksp))
> fprintf(fp, "%0lx: Task is running in user space\n",
> bt_in->task);
> else
> fprintf(fp, "%0lx: Invalid Stack Pointer %0lx\n",
> bt_in->task, *ksp);
> *nip = pt_regs->nip;
> ppc64_print_regs(pt_regs);
> return FALSE;
> }
> }
>
> And if the task were to have been running in userspace, it already dumps the
> registers in the "else" section above.
>
> I see that the regs->trap is 0, so I understand now that there's nothing to
> translate w/respect to the exception frame type, but a follow-up translation
> of the NIP and LR would at least show that there was some kind of hypercall
> involved. (Whether it can be firmly determined whether FADUMP was responsible
> is another question)
>
>
Hi Dave,
I did think of it but I was wary considering two register prints like below,
if there is an exception frame..
PID: 2121 TASK: c0000001af90c600 CPU: 2 COMMAND: "sshd"
R0: c0000000003e5280 R1: c0000001ae047a30 R2:
c000000000fd5a00
R3: 0000000000000001 R4: 000000000000019e R5:
000000000000000f
R6: 0000000000000004 R7: c0000001ae047bb8 R8:
00000000000b3d9f
R9: 00000000000000f0 R10: 0000000000000678 R11:
c0000000008e0f38
R12: c0000000003e6310 R13: c00000000b781200 R14:
0000000000000000
R15: 0000000000000000 R16: 000001000b7dad70 R17:
000000005dfd3c08
R18: 000000005dfd2838 R19: 00003ffff81eb620 R20:
000000005df74128
R21: 000001000b7d89a0 R22: 000000000000de4c R23:
000000005df73b30
R24: 000000005dfd3c88 R25: 00003ffff81eb428 R26:
c0000001ae047bb8
R27: c0000001b17f4d80 R28: c000000000c60580 R29:
000000000000019e
R30: 000000000000000f R31: 000000000000090b
NIP: 00003fffb6ac8400 MSR: 800000000000d033 OR3:
0000000000000000
CTR: c0000000003e6310 LR: c0000000003e493c XER:
0000000020000000
CCR: 0000000024004824 MQ: 0000000000000000 DAR:
000001000b7e1640
DSISR: 0000000002000000 Syscall Result: 0000000000000000
#0 [c0000001ae047a30] (null) at c0000000fd783c00 (unreliable)
#1 [c0000001ae047a70] avc_has_perm at c0000000003e5280
#2 [c0000001ae047b60] sock_has_perm at c0000000003e6238
#3 [c0000001ae047be0] security_socket_sendmsg at c0000000003e28fc
#4 [c0000001ae047c30] sock_sendmsg at c00000000072d53c
#5 [c0000001ae047c60] sock_write_iter at c00000000072d644
#6 [c0000001ae047d00] __vfs_write at c0000000002ed97c
#7 [c0000001ae047d90] vfs_write at c0000000002ef328
#8 [c0000001ae047de0] sys_write at c0000000002f0f00
#9 [c0000001ae047e30] system_call at c00000000000b184
System Call [c00] exception frame:
R0: 0000000000000004 R1: 00003ffff81eb220 R2:
00003fffb6b99800
R3: 0000000000000003 R4: 000001000b80e3c0 R5:
0000000000000034
R6: 00003ffff81eb2e4 R7: 000000000000021e R8:
0000000000000000
R9: 0000000000000000 R10: 0000000000000000 R11:
0000000000000000
R12: 0000000000000000 R13: 00003fffb6497730 R14:
0000000000000000
R15: 0000000000000000 R16: 000001000b7dad70 R17:
000000005dfd3c08
R18: 000000005dfd2838 R19: 00003ffff81eb620 R20:
000000005df74128
R21: 000001000b7d89a0 R22: 000000000000de4c R23:
000000005df73b30
R24: 000000005dfd3c88 R25: 00003ffff81eb428 R26:
00003ffff81eb430
R27: 00003ffff81eb420 R28: 00003ffff81eb424 R29:
00003ffff81eb2e4
R30: 000001000b80e3c0 R31: 0000000000000034
NIP: 00003fffb6ac8400 MSR: 800000000000d033 OR3:
0000000000000003
CTR: 0000000000000000 LR: 000000005df1c3e4 XER:
0000000000000000
CCR: 0000000044004824 MQ: 0000000000000001 DAR:
00003fffb729c590
DSISR: 000000000a000000 Syscall Result: 0000000000000000
instead of this..
PID: 2121 TASK: c0000001af90c600 CPU: 2 COMMAND: "sshd"
#0 [c0000001ae047a30] (null) at c0000000fd783c00 (unreliable)
#1 [c0000001ae047a70] avc_has_perm at c0000000003e5280
#2 [c0000001ae047b60] sock_has_perm at c0000000003e6238
#3 [c0000001ae047be0] security_socket_sendmsg at c0000000003e28fc
#4 [c0000001ae047c30] sock_sendmsg at c00000000072d53c
#5 [c0000001ae047c60] sock_write_iter at c00000000072d644
#6 [c0000001ae047d00] __vfs_write at c0000000002ed97c
#7 [c0000001ae047d90] vfs_write at c0000000002ef328
#8 [c0000001ae047de0] sys_write at c0000000002f0f00
#9 [c0000001ae047e30] system_call at c00000000000b184
System Call [c00] exception frame:
R0: 0000000000000004 R1: 00003ffff81eb220 R2:
00003fffb6b99800
R3: 0000000000000003 R4: 000001000b80e3c0 R5:
0000000000000034
R6: 00003ffff81eb2e4 R7: 000000000000021e R8:
0000000000000000
R9: 0000000000000000 R10: 0000000000000000 R11:
0000000000000000
R12: 0000000000000000 R13: 00003fffb6497730 R14:
0000000000000000
R15: 0000000000000000 R16: 000001000b7dad70 R17:
000000005dfd3c08
R18: 000000005dfd2838 R19: 00003ffff81eb620 R20:
000000005df74128
R21: 000001000b7d89a0 R22: 000000000000de4c R23:
000000005df73b30
R24: 000000005dfd3c88 R25: 00003ffff81eb428 R26:
00003ffff81eb430
R27: 00003ffff81eb420 R28: 00003ffff81eb424 R29:
00003ffff81eb2e4
R30: 000001000b80e3c0 R31: 0000000000000034
NIP: 00003fffb6ac8400 MSR: 800000000000d033 OR3:
0000000000000003
CTR: 0000000000000000 LR: 000000005df1c3e4 XER:
0000000000000000
CCR: 0000000044004824 MQ: 0000000000000001 DAR:
00003fffb729c590
DSISR: 000000000a000000 Syscall Result: 0000000000000000
On second thought, that may not be bad after all??
So, I am ok with the change you propose.
Thanks
Hari
More information about the Crash-utility
mailing list