[Crash-utility] [ANNOUNCE][RFC] gcore extension module: user-mode process core dump

HATAYAMA Daisuke d.hatayama at jp.fujitsu.com
Tue Feb 1 00:25:14 UTC 2011


Hello Dave,

I've downloaded vmcore and vmlinux. Thanks a lot.

From: Dave Anderson <anderson at redhat.com>
Subject: Re: [ANNOUNCE][RFC] gcore extension module: user-mode process core dump
Date: Mon, 31 Jan 2011 08:51:04 -0500 (EST)

> 
> Hello Daisuke,
> 
> The test dump can be found here:
> 
>   http://people.redhat.com/anderson/.gcore_test_dump
> 
> One important thing to note -- the dumpfile was taken with
> the "snap.so" extension module while running live.  It
> selects the "crash" process that was doing the live dump
> as the panic task.  So when you do a backtrace on it, it
> looks like this: 
> 
> crash> bt
> PID: 2080   TASK: ffff880079ed2480  CPU: 0   COMMAND: "crash"
>  #0 [ffff88007a615b08] schedule at ffffffff81480533
>  #1 [ffff88007a615bf0] rcu_read_unlock at ffffffff811edfd3
>  #2 [ffff88007a615c00] avc_has_perm_noaudit at ffffffff811eea76
>  #3 [ffff88007a615c90] avc_has_perm at ffffffff811eeae3
>  #4 [ffff88007a615d10] inode_has_perm at ffffffff811f2815
>  #5 [ffff88007a615de8] might_fault at ffffffff810f22ec
>  #6 [ffff88007a615e80] might_fault at ffffffff810f2335
>  #7 [ffff88007a615eb0] crash_read at ffffffffa004f103 [crash]
>  #8 [ffff88007a615f00] vfs_read at ffffffff8112115b
>  #9 [ffff88007a615f40] sys_read at ffffffff81121278
> #10 [ffff88007a615f80] system_call_fastpath at ffffffff81009c72
>     RIP: 000000333a0d41b0  RSP: 00007fffac23a7f0  RFLAGS: 00000206
>     RAX: 0000000000000000  RBX: ffffffff81009c72  RCX: 0000000000000000
>     RDX: 0000000000001000  RSI: 0000000000ca5440  RDI: 0000000000000004
>     RBP: 0000000000000004   R8: 000000007a615000   R9: 0000000000000006
>     R10: 00000000fffffff8  R11: 0000000000000246  R12: 0000000000ca5440
>     R13: 0000000000001000  R14: 0000000000001000  R15: 000000007a615000
>     ORIG_RAX: 0000000000000000  CS: 0033  SS: 002b
> crash>
> 
> Now, when using "snap.so" to create a dumpfile, all of the "active"
> backtraces are not legitimate, because they were *running* when their
> kernel stacks were being read.  So, for example, the "snap.so" code
> was running -- doing a read() -- when the "crash" stack was read.  But 
> since it had not panicked, there were no legitimate starting RIP/RSP
> values to use for starting points for the backtrace.  So frame #'s 0 
> through #7 above should not be accepted as "real".  But I presume that
> starting from frame #7 , would be correct.  

Ah, there's no method to obtain active registers...

If register values is unavailable for an active task, gcore is now
treating it in the same way as for a sleeping task. This means gcore
chooses RIP and RSP the scheduler saved last time.

Applying this story to here, it seems to me that the old logic of
resotre_frame_pointer() can surely result in non-termination around
the frame #7, since at the point old stack frame is switching to new
one and a list of frame pointers is not connected.

I'll now verify this story by looking at vmcore you gave me.

Thanks,
HATAYAMA Daisuke




More information about the Crash-utility mailing list