[Crash-utility] enhance bt command

[Dave Anderson]

Ming Zhang wrote:
> On Mon, 2008-03-03 at 15:35 -0500, Dave Anderson wrote:
> 
> <snip>
> 
>>>>> so we know the object address, which slab it is in, and the offset,
>>>>> (thus can derive the raw value), all in one shot?
>>>>>
>>>> You've done a fine bit of debugging your issue...
>>>>
>>>> But I think that's a bit of overkill for each address reference.
>>>>
>>>> To do it right it would have to walk slab chains to gather all of the
>>>> information needed by the "kmem -S" output, which can be extremely
>>>> time-consuming, and potentially error-prone if the slab chain is
>>>> corrupt or being modified while running on a live system.
>>> full agree. realized now.
>>>
>>> then here is my question, how "kmem -s <addr>" find out which slab this
>>> address belong to? by only looking at the page?
>> The kmem_cache and containing slab addresses are stashed in unused
>> fields of the page structure of the object's virtual address.
> 
> ic. good to know this.
> 
> 
>>> then here is my revised suggestion. can we split this into 2 steps?
>>>
>>> (1) rd -S show [raw address: cache name]
>>>
>>> (this is a great to have since do rd 2 times, one with -S and another
>>> without -S is tedious.)
>> But in your case, for example, I'm presuming you've done a "bt -f",
>> and simply would like a symbols/slab translation of a chunk of memory
>> making up one of the stack frames, so you do a subsequent "rd -S" of
>> that memory area.  Doesn't seem that tedious to me...
> 
> have to agree.
> 
> 
>>> (2) and kmem -s <address>
>>>
>>> show which slab it is in and optionally (with a new option like -O) show
>>> the object?
>> Not following -- "kmem -s <address>" does show which slab it's in.
> 
> i emphasize on "and optionally...", ;)
> 
> 
>> And if you're asking whether the object can be dumped as the structure
>> type it is (?), well, how would it be possible from the crash utility's
>> viewpoint to even know what type of data structure it is?
> 
> no. what i mean is kmem -s <address> does not give me the object address
> 
> crash> kmem -s 000001007877c8d4
> CACHE            NAME                 OBJSIZE  ALLOCATED     TOTAL
> SLABS  SSIZE
> 10037ffc080      dentry_cache             240       9429     10560
> 660     4k
> SLAB              MEMORY            TOTAL  ALLOCATED  FREE
> 1007877c040       1007877c088          16          9     7
> FREE / [ALLOCATED]
>   [1007877c8d4]
> 
> 
> i still have to use kmem -S to find out 1007877c808 is the object that
> contain this address. nice if crash can do this for me.

OK I understand.  Yeah, it's always worked that way -- I don't recall
why other than the fact that by the time the address is displayed, the
function that does the display no longer has a handle on the beginning
address of the object, only the "requested" address, the slab it came
from, whether it's free/allocated, and whether it's sitting on a per-cpu
cache.  I'll have to revisit that sometime...

> 
> 
>> If you want to look at all of the objects in a slab represented
>> as data structures, you're going to have supply the knowledge of
>> what data structure they are.  It's simple enough, just do a "kmem -S"
>> into a file, delete everything except the object addresses that you're
>> interested in, insert "struct whatever" in front of each address, save
> 
> this is exactly what i did when i have to do work like this by using
> gawk, tr, and grep.
> 
>> the file -- and run it as crash input file.
>>
> 
> how to do this? i know crash -i can run a file at beginning. but how to
> run command in a file at any moment?
> 

Enter "help input" -- where it talks about "<":

   crash> help input
   ...
   Lastly, a set of crash commands may be entered into a regular file that can
   used as input, again using standard command line syntax:

     crash> < inputfile
   ...
   crash>

For example:

   crash> !cat /tmp/input
   bt
   sys
   set
   crash> < /tmp/input
   crash> bt
   PID: 1      TASK: c148eaa0  CPU: 1   COMMAND: "init"
    #0 [c148db04] schedule at c0604141
    #1 [c148db6c] schedule_timeout at c060487f
    #2 [c148db90] do_select at c04800b7
    #3 [c148de34] core_sys_select at c04803ba
    #4 [c148df74] sys_select at c0480981
    #5 [c148dfb8] system_call at c0404ef8
       EAX: 0000008e  EBX: 0000000b  ECX: bfc96b30  EDX: 00000000
       DS:  007b      ESI: 00000000  ES:  007b      EDI: bfc96c60
       SS:  007b      ESP: bfc96afc  EBP: bfc96df8
       CS:  0073      EIP: 00a4c402  ERR: 0000008e  EFLAGS: 00000246
   crash> sys
         KERNEL: /usr/lib/debug/lib/modules/2.6.18-53.el5/vmlinux
       DUMPFILE: /dev/crash
           CPUS: 2
           DATE: Mon Mar  3 16:11:57 2008
         UPTIME: 20 days, 06:18:19
   LOAD AVERAGE: 0.00, 0.02, 0.05
          TASKS: 176
       NODENAME: crash.boston.redhat.com
        RELEASE: 2.6.18-53.el5
        VERSION: #1 SMP Wed Oct 10 16:34:02 EDT 2007
        MACHINE: i686  (1993 Mhz)
         MEMORY: 511.5 MB
   crash> set
       PID: 1
   COMMAND: "init"
      TASK: c148eaa0  [THREAD_INFO: c148d000]
       CPU: 1
     STATE: TASK_INTERRUPTIBLE
   crash>

Dave