[Crash-utility] Viewing module symbols using crash

[Luc Chouinard]

Embedded symbolic information is always the best source, especially for
types. 

But before going completely primitive and scanning through hexadecimal
dumps, you might want to consider using the C interpreter (sial) to
define you types and to access and print your data. 

As long as you have a start address for your data (maybe from the
associated kernel module entry, or some print out at the console) and
are respecting the -D definitions that could impact the type definition
itself, you should be able to define (i.e. cut&paste) types and walk the
driver structures even in cases where symbolic information is completely
missing.

The interpreter has a couple of built-in primitives that can print types
and typed values (something like print(typed value|type)) if I remember
correctly. Somewhat akin to a pretty-print output.

   -Luc

On Fri, 2008-08-08 at 09:32 -0400, Dave Anderson wrote:
> Durga Prasad wrote:
> > Hi,
> > 
> > [1]:
> > I need to debug a kernel module (a device driver).
> > I added the desired module to crash using mod -S <path to module>. The 
> > driver is compiled with -g option.
> > I could not see the structures defined within the module.
> > 
> > Am I missing something?
> 
> Accessing data in kernel modules can be spotty at best,
> but the embedded gdb module should typically be able
> to pick up structure definitions in modules.
> 
> For example, these two structures are defined in ./fs/ext3/xattr.c
> like so:
> 
>    struct ext3_xattr_info {
>            int name_index;
>            const char *name;
>            const void *value;
>            size_t value_len;
>    };
> 
>    struct ext3_xattr_search {
>            struct ext3_xattr_entry *first;
>            void *base;
>            void *end;
>            struct ext3_xattr_entry *here;
>            int not_found;
>    };
> 
> Without the ext3 module debuginfo, crash (gdb) doesn't know what
> they are:
> 
>    # crash -s
>    crash> struct ext3_xattr_search
>    struct: invalid data structure reference: ext3_xattr_search
>    crash> struct ext3_xattr_info
>    struct: invalid data structure reference: ext3_xattr_info
>    crash>
> 
> Until after adding the module debuginfo data:
> 
>    crash> mod -s ext3
>         MODULE       NAME               SIZE  OBJECT FILE
>    ffffffff8806be00  ext3             168017  /lib/modules/2.6.18-102.el5.perfmon2/kernel/fs/ext3/ext3.ko
>    crash> struct ext3_xattr_search
>    struct ext3_xattr_search {
>        struct ext3_xattr_entry *first;
>        void *base;
>        void *end;
>        struct ext3_xattr_entry *here;
>        int not_found;
>    }
>    SIZE: 40
>    crash> struct ext3_xattr_info
>    struct ext3_xattr_info {
>        int name_index;
>        const char *name;
>        const void *value;
>        size_t value_len;
>    }
>    SIZE: 32
>    crash>
> 
> Then you may be able to "p" individual instances of the structures
> if there are symbolic/static declaration of them.  Otherwise you
> can always cast the address of the structure to the structure type,
> as in:
> 
>    crash> struct ext3_xattr_info <address>
> 
> If you cannot get the structure declaration after using "mod -[sS]",
> then you'll have to get primitive, and just dump the data at the
> relevant address with "rd".
> 
> > 
> > System info:
> >     KERNEL: vmlinux.sles10sp2.smp.x86_64
> >     DUMPFILE: 2008-08-08-18:57/vmcore
> >     CPUS: 4
> > 
> >      RELEASE: 2.6.16.60-0.21-smp
> >      VERSION: #1 SMP Tue May 6 12:41:02 UTC 2008
> >      MACHINE: x86_64  (1995 Mhz)
> >      MEMORY: 3.9 GB
> > 
> > 
> > 
> > [2]:
> > I could get into live kernel debugging using crash on RHEL 5 after 
> > installing relevant kernel-debug packages. How do I do the same on SLES 10?
> > As of now, I have been compiling the kernel with CONFIG_DEBUG_INFO set. 
> > I can use the resulting vmlinux for post crash analysis, but not for 
> > live analysis.  Any light on how to accomplish this would be appreciated.
> 
> For SLES I'm guessing that you could use the post-built (CONFIG_DEBUG_INFO) kernel,
> but also add the System.map file of the live kernel to the command line, as in:
> 
>    $ crash /boot/System.map vmlinux-post-built-with-debuginfo
> 
> Crash will use the debuginfo data from the vmlinux file, but then go in
> and back-patch all the vmlinux file's minimal_symbols with the correct
> symbol values from the System.map file.
> 
> But Bernhard Walle (bwalle at suse.de) can answer that part of your question.
> He's active on this list, but I've added him to the cc: list to make
> sure he sees this.
> 
> Dave Anderson
> 
> 
> > 
> > Thanks
> > Durga Prasad
> > 
> 
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>