[Crash-utility] Module load patch for crash-4.0-3.17
Dave Anderson
anderson at redhat.com
Wed Jan 17 22:41:11 UTC 2007
Hi Castor,
Preliminary tests look good on x86_64, and for the most part ia64.
On ia64, some modules never load any .data, .data.rel, etc. sections,
and when that happens the "p" command for their data (d) symbols
doesn't work. But other modules work just fine? No regression
though...
On s390x, we don't stand a prayer, because the modules themselves
don't seem to contain any debug info? It looks like this:
crash> mod -l
MODULE NAME SIZE OBJECT FILE
2081a800 dm_mirror 56184 (not loaded) [CONFIG_KALLSYMS]
20826b00 sd_mod 44584 (not loaded) [CONFIG_KALLSYMS]
2082f100 dasd_fba_mod 29696 (not loaded) [CONFIG_KALLSYMS]
2084c600 jbd 100688 (not loaded) [CONFIG_KALLSYMS]
20867900 dasd_mod 116892 (not loaded) [CONFIG_KALLSYMS]
20880600 dasd_eckd_mod 84224 (not loaded) [CONFIG_KALLSYMS]
2088d900 dm_zero 19456 (not loaded) [CONFIG_KALLSYMS]
208c0b00 ext3 200208 (not loaded) [CONFIG_KALLSYMS]
208e9600 scsi_transport_fc 67848 (not loaded) [CONFIG_KALLSYMS]
208f4e00 dm_snapshot 41288 (not loaded) [CONFIG_KALLSYMS]
2094b900 scsi_mod 233104 (not loaded) [CONFIG_KALLSYMS]
20966100 qdio 103768 (not loaded) [CONFIG_KALLSYMS]
2097f900 dm_mod 99544 (not loaded) [CONFIG_KALLSYMS]
20993c00 ccwgroup 73472 (not loaded) [CONFIG_KALLSYMS]
209a5300 cu3088 67080 (not loaded) [CONFIG_KALLSYMS]
209f5a00 zfcp 236832 (not loaded) [CONFIG_KALLSYMS]
20a4ed00 sg 151600 (not loaded) [CONFIG_KALLSYMS]
20bcbf00 autofs4 254472 (not loaded) [CONFIG_KALLSYMS]
20c67600 qeth 563600 (not loaded) [CONFIG_KALLSYMS]
20fb0000 sunrpc 1076664 (not loaded) [CONFIG_KALLSYMS]
212aa100 ipv6 2300920 (not loaded) [CONFIG_KALLSYMS]
crash> mod -s ext3
MODULE NAME SIZE OBJECT FILE
208c0b00 ext3 200208 /lib/modules/2.6.18-1.2961.el5/kernel/fs/ext3/ext3.ko
crash> p ext3_file_operations
ext3_file_operations = $1 = 0
crash> whatis ext3_file_operations
<data variable, no debug info> ext3_file_operations;
crash> sym ext3_file_operations
208bff88 (D) ext3_file_operations
crash>
The add-symbol-file line looks OK, although the address of
ext3_file_operations looks like it's in the .data.rel.ro_addr
section (?):
add-symbol-file request add-symbol-file /lib/modules/2.6.18-1.2961.el5/kernel/fs/ext3/ext3.ko 0x208953a8 -s .exit.text
0x208adfc4 -s .rodata 0x208ae000 -s .rodata.str1.2 0x208af1b8 -s __versions 0x208b1d60 -s .data.rel.ro 0x208bfea0 -s
.data.rel 0x208c01f8 -s .data.rel.local 0x208c0740 -s .gnu.linkonce.this_module 0x208c0b00 -s .bss 0x208c4e00
add symbol table from file "/lib/modules/2.6.18-1.2961.el5/kernel/fs/ext3/ext3.ko" at
.text_addr = 0x208953a8
.exit.text_addr = 0x208adfc4
.rodata_addr = 0x208ae000
.rodata.str1.2_addr = 0x208af1b8
__versions_addr = 0x208b1d60
.data.rel.ro_addr = 0x208bfea0
.data.rel_addr = 0x208c01f8
.data.rel.local_addr = 0x208c0740
.gnu.linkonce.this_module_addr = 0x208c0b00
.bss_addr = 0x208c4e00
Anyway, the behavior is identical to what it's always been,
so there's no regression.
I'll keep tinkering as I get time...
Thanks,
Dave
Castor Fu wrote:
>
> Finding the overrun wasn't actually that hard. It's also fixed in the current GDB tree.
> I've attached a patch which fixes the problem in symfile.c. The point of this patch
> is to fix loading kernel modules symbol information on 2.6 for those who have not
> been following this.
> Hopefully this will work on other platforms too....
> -castor
> -----------------------------------------------------------------------------------------------------------------------
> From: crash-utility-bounces at redhat.com [mailto:crash-utility-bounces at redhat.com] On Behalf Of Castor Fu
> Sent: Wednesday, January 17, 2007 7:49 AM
> To: Discussion list for crash utility usage, maintenance and development; Discussion list for crash utility usage,
> maintenance and development
> Subject: RE: [Crash-utility] test results of latest 4.0-3.16.sym.patch (ia64)
>
>
> Hi Dave:
>
> I reproduced the problem on an x86 system by creating a module with a bunch
> of sections.
>
> I then found the following in gdb-6.1/gdb/symfile.c:add_symbol_table_command()
>
>
> num_sec_opts = 16;
>
> with additional code for xreallocing if it turned out to have too many
> sections.
>
> This seems to be the code which is broken. I'm loath to figure out exactly
> what it is... I'll put together a patch against 4.0-3.17 which jacks this
> up, and probably print a warning if we exceed the count.
>
> Thanks for digging so far into this.
>
> -castor
>
> -----Original Message-----
> From: crash-utility-bounces at redhat.com on behalf of Dave Anderson
> Sent: Thu 1/4/2007 8:22 AM
> To: Discussion list for crash utility usage, maintenance and development
> Subject: Re: [Crash-utility] test results of latest 4.0-3.16.sym.patch (ia64)
>
>
> Hi Castor,
>
> Another FYI re: the xrealloc() crash. The problem appears
> to be specific to gdb.
>
> I captured the "add-symbol-file" command string and saved
> it in an input file. Then I brought crash up and executed
> the input file, which simply passes the suspect command line
> directly to gdb, and it crashes on its own:
>
> crash> < /tmp/junk
> crash> add-symbol-file /lib/modules/2.6.18-1.2767.el5/kernel/net/ipv6/ipv6.ko 0xa00000021ed605b0 -s .exit.text
> 0xa00000021edb49a0 -s .rodata 0xa00000021edbd4c8 -s __ksymtab_strings 0xa00000021edbdc08 -s __versions
> 0xa00000021edbdf98 -s .data 0xa00000021edd6a20 -s .data.rel.ro 0xa00000021edd6c00 -s __ksymtab_gpl 0xa00000021edd6df8
> -s __kcrctab_gpl 0xa00000021edd6ed8 -s .data.rel 0xa00000021edd6f48 -s .data.rel.local 0xa00000021ee39940 -s
> .data.rel.ro.local 0xa00000021ee3a9c0 -s .data.read_mostly 0xa00000021ee3a9e0 -s __ksymtab 0xa00000021ee3aa60 -s
> __kcrctab 0xa00000021ee3ac30 -s .gnu.linkonce.this_module 0xa00000021ee3ad80 -s .sdata 0xa00000021ee5d730 -s .bss
> 0xa00000021ee5b000 -s .sbss 0xa00000021ee5e8b8
> add_symbol_file_command: calling xrealloc w/argcnt: 49 arg: [0xa00000021ee5d730]...
> *** glibc detected *** ./crash: realloc(): invalid next size: 0x6000000001921fe0 ***
> ======= Backtrace: =========
> /lib/libc.so.6.1[0x20000000002f2a70]
> /lib/libc.so.6.1(realloc-0x1cb0b0)[0x20000000002f5e20]
> ./crash(xmrealloc+0x1fffffffffee6e20)[0x40000000003a7d00]
> ./crash[0x40000000002ff500]
> ./crash[0x40000000004221e0]
> ./crash(cmd_func+0x1ffffffffff61610)[0x4000000000422500]
> ./crash(execute_command+0x1fffffffffee25f0)[0x40000000003a34f0]
> ./crash(gdb_command_funnel+0x1fffffffffe2feb0)[0x40000000002f0dc0]
> ./crash(gdb_interface+0x1fffffffffcd7590)[0x40000000001984b0]
> ./crash(gdb_pass_through+0x1fffffffffcd6cb0)[0x4000000000197be0]
> ./crash(cmd_gdb+0x2000000000151068)[0x400000000019bbc0]
> ./crash(exec_command+0x1fffffffffb99db0)[0x400000000005acf0]
> ./crash(exec_input_file+0x1fffffffffd86d40)[0x4000000000247c90]
> ./crash[0x400000000005b420]
> ./crash(exec_command+0x1fffffffffb99e50)[0x400000000005ad90]
> ./crash(main_loop+0x1fffffffffb9a2e0)[0x400000000005a8e0]
> ./crash(current_interp_command_loop+0x200000000001fd60)[0x40000000004e0cc0]
> ./crash[0x40000000003199c0]
> ./crash[0x400000000039f370]
> ./crash[0x40000000003a4260]
> ./crash(catch_errors+0x1fffffffffee33b0)[0x40000000003a4320]
> ./crash[0x400000000031a930]
> ./crash[0x400000000039f370]
> ./crash[0x40000000003a4260]
> ./crash(catch_errors+0x1fffffffffee33b0)[0x40000000003a4320]
> ./crash(gdb_main+0x1fffffffffe58960)[0x40000000003198e0]
> ./crash(gdb_main_entry+0x1fffffffffe589f0)[0x4000000000319980]
> ./crash(gdb_main_loop+0x1fffffffffcd54d0)[0x4000000000196470]
> ./crash(main+0x1fffffffffb99820)[0x400000000005a330]
> /lib/libc.so.6.1(__libc_start_main-0x2818f0)[0x200000000023f6c0]
> ./crash(_start+0x1fffffffffb95240)[0x4000000000056200]
> ======= Memory map: ========
> 00000000-00004000 r--p 00000000 00:00 0
> 2000000000000000-2000000000038000 r-xp 00000000 fd:00 10256390 /lib/ld-2.5.so
> 2000000000044000-2000000000050000 rw-p 00034000 fd:00 10256390 /lib/ld-2.5.so
> 2000000000050000-2000000000114000 r-xp 00000000 fd:00 10256405 /lib/libm-2.5.so
> 2000000000114000-2000000000120000 ---p 000c4000 fd:00 10256405 /lib/libm-2.5.so
> 2000000000120000-2000000000124000 rw-p 000c0000 fd:00 10256405 /lib/libm-2.5.so
> 2000000000124000-20000000001b0000 r-xp 00000000 fd:00 10883077 /usr/lib/libncurses.so.5.5
> 20000000001b0000-20000000001bc000 ---p 0008c000 fd:00 10883077 /usr/lib/libncurses.so.5.5
> 20000000001bc000-20000000001cc000 rw-p 00088000 fd:00 10883077 /usr/lib/libncurses.so.5.5
> 20000000001cc000-20000000001d0000 rw-p 20000000001cc000 00:00 0
> 20000000001d0000-20000000001d8000 r-xp 00000000 fd:00 10256403 /lib/libdl-2.5.so
> 20000000001d8000-20000000001e4000 ---p 00008000 fd:00 10256403 /lib/libdl-2.5.so
> 20000000001e4000-20000000001e8000 rw-p 00004000 fd:00 10256403 /lib/libdl-2.5.so
> 20000000001e8000-200000000020c000 r-xp 00000000 fd:00 10882711 /usr/lib/libz.so.1.2.3
> 200000000020c000-2000000000218000 ---p 00024000 fd:00 10882711 /usr/lib/libz.so.1.2.3
> 2000000000218000-200000000021c000 rw-p 00020000 fd:00 10882711 /usr/lib/libz.so.1.2.3
> 200000000021c000-2000000000480000 r-xp 00000000 fd:00 10256397 /lib/libc-2.5.so
> 2000000000480000-200000000048c000 ---p 00264000 fd:00 10256397 /lib/libc-2.5.so
> 200000000048c000-2000000000498000 rw-p 00260000 fd:00 10256397 /lib/libc-2.5.so
> 2000000000498000-20000000004d8000 rw-p 2000000000498000 00:00 0
> 20000000004d8000-2000000003c1c000 r--p 00000000 fd:00 10882710 /usr/lib/locale/locale-archive
> 2000000003c1c000-2000000003c2c000 rw-p 2000000003c1c000 00:00 0
> 2000000003c38000-2000000003c44000 r-xp 00000000 fd:00 10256427 /lib/libthread_db-1.0.so
> 2000000003c44000-2000000003c50000 ---p 0000c000 fd:00 10256427 /lib/libthread_db-1.0.so
> 2000000003c50000-2000000003c54000 rw-p 00008000 fd:00 10256427 /lib/libthread_db-1.0.so
> 2000000003c54000-2000000003c58000 rw-p 2000000003c54000 00:00 0
> 2000000003c6c000-2000000003da0000 rw-p 2000000003c6c000 00:00 0
> 2000000003da0000-2000000003dbc000 r-xp 00000000 fd:00 10884674 /usr/lib/libunwind.so.7.0.0
> 2000000003dbc000-2000000003dc8000 ---p 0001c000 fd:00 10884674 /usr/lib/libunwind.so.7.0.0
> 2000000003dc8000-2000000003dcc000 rw-p 00018000 fd:00 10884674 /usr/lib/libunwind.so.7.0.0
> 2000000003dcc000-2000000003df0000 rw-p 2000000003dcc000 00:00 0
> 2000000003e00000-2000000003e08000 r--s 00000000 fd:00 10977539 /usr/lib/gconv/gconv-modules.cache
> 2000000003e08000-2000000003e18000 rw-p 2000000003e08000 00:00 0
> 2000000003e1c000-2000000006edc000 rw-p 2000000003e1c000 00:00 0
> 2000000006ee8000-2000000006f04000 r-xp 00000000 fd:00 10256386 /lib/libgcc_s-4.1.1-20061130.so.1
> 2000000006f04000-2000000006f10000 ---p 0001c000 fd:00 10256386 /lib/libgcc_s-4.1.1-20061130.so.1
> 2000000006f10000-2000000006f14000 rw-p 00018000 fd:00 10256386 /lib/libgcc_s-4.1.1-20061130.so.1
> 2000000006f14000-2000000006f24000 rw-p 2000000006f14000 00:00 0
> 2000000008000000-2000000008024000 rw-p 2000000008000000 00:00 0
> 2000000008024000-200000000c000000 ---p 2000000008024000 00:00 0
> 4000000000000000-40000000007e0000 r-xp 00000000 fd:00 9633909 /var/tmp/crash-4.0-3.16/crash
> 600000000000c000-600000000006c000 rw-p 007dc000 fd:00 9633909 /var/tmp/crash-4.0-3.16/crash
> 600000000006c000-6000000001fc0000 rw-p 600000000006c000 00:00 0 [heap]
> 60000fff7fffc000-60000fff80004000 rw-p 60000fff7fffc000 00:00 0
> 60000ffffe068000-60000ffffe0bc000 rw-p 60000ffffe068000 00:00 0 [stack]
> a000000000000000-a000000000020000 ---p 00000000 00:00 0 [vdso]
> Aborted
>
>
>
>
> -----------------------------------------------------------------------------------------------------------
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/crash-utility/attachments/20070117/8d02b8e2/attachment.htm>
More information about the Crash-utility
mailing list