[Crash-utility] crash-6.0.2 won't do module source lines on our kernel
Dave Anderson
anderson at redhat.com
Tue Jan 31 15:16:38 UTC 2012
----- Original Message -----
>
> So 5.1.9 works, and 6.0.2 fails if I load all modules with mod -S.
> That's interesting.
>
> Bob Montgomery
Strange -- here "mod -S" alone works OK on a freshly-installed F16 kernel:
# /usr/bin/crash
crash 6.0.2-1.fc16
Copyright (C) 2002-2011 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/vmlinux
DUMPFILE: /dev/crash
CPUS: 4
DATE: Tue Jan 31 09:51:19 2012
UPTIME: 00:43:03
LOAD AVERAGE: 0.13, 0.14, 0.19
TASKS: 110
NODENAME: dell-pec5125-02.lab.bos.redhat.com
RELEASE: 3.2.2-1.fc16.x86_64
VERSION: #1 SMP Thu Jan 26 03:21:58 UTC 2012
MACHINE: x86_64 (2599 Mhz)
MEMORY: 2 GB
PID: 32235
COMMAND: "crash"
TASK: ffff88007ba8ae40 [THREAD_INFO: ffff88003e310000]
CPU: 3
STATE: TASK_RUNNING (ACTIVE)
crash> mod -S
MODULE NAME SIZE OBJECT FILE
ffffffffa0002420 pata_atiixp 13197 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/pata_atiixp.ko
ffffffffa000a5c0 ata_generic 12899 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/ata_generic.ko
ffffffffa0012420 pata_acpi 13027 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/pata_acpi.ko
ffffffffa0041580 sunrpc 235041 /lib/modules/3.2.2-1.fc16.x86_64/kernel/net/sunrpc/sunrpc.ko
ffffffffa0055360 dcdbas 14828 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/firmware/dcdbas.ko
ffffffffa005a1e0 k10temp 13119 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/hwmon/k10temp.ko
ffffffffa00600a0 joydev 17412 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/input/joydev.ko
ffffffffa0069140 edac_mce_amd 22882 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/edac_mce_amd.ko
ffffffffa0073f20 edac_core 51619 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/edac_core.ko
ffffffffa007c060 dca 14601 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/dca/dca.ko
ffffffffa00834c0 amd64_edac_mod 23795 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/amd64_edac_mod.ko
ffffffffa008c600 i2c_core 37955 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/i2c/i2c-core.ko
ffffffffa0093cc0 i2c_piix4 13694 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/i2c/busses/i2c-piix4.ko
ffffffffa0098120 sp5100_tco 13697 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/watchdog/sp5100_tco.ko
ffffffffa009d140 crash 12576 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/char/crash.ko
ffffffffa00a7380 microcode 23240 /lib/modules/3.2.2-1.fc16.x86_64/kernel/arch/x86/kernel/microcode.ko
ffffffffa00ba9a0 lockd 84578 /lib/modules/3.2.2-1.fc16.x86_64/kernel/fs/lockd/lockd.ko
ffffffffa00e3000 igb 147890 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/net/ethernet/intel/igb/igb.ko
crash> dis -l rpc_sleep_on_priority 10
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/net/sunrpc/sched.c: 351
0xffffffffa0022650 <rpc_sleep_on_priority>: push %rbp
0xffffffffa0022651 <rpc_sleep_on_priority+1>: mov %rsp,%rbp
0xffffffffa0022654 <rpc_sleep_on_priority+4>: sub $0x20,%rsp
0xffffffffa0022658 <rpc_sleep_on_priority+8>: mov %rbx,-0x20(%rbp)
0xffffffffa002265c <rpc_sleep_on_priority+12>: mov %r12,-0x18(%rbp)
0xffffffffa0022660 <rpc_sleep_on_priority+16>: mov %r13,-0x10(%rbp)
0xffffffffa0022664 <rpc_sleep_on_priority+20>: mov %r14,-0x8(%rbp)
0xffffffffa0022668 <rpc_sleep_on_priority+24>: data32 data32 data32 xchg %ax,%ax
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/arch/x86/include/asm/bitops.h: 312
0xffffffffa002266d <rpc_sleep_on_priority+29>: mov 0x70(%rsi),%rax
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/net/sunrpc/sched.c: 351
0xffffffffa0022671 <rpc_sleep_on_priority+33>: mov %rdi,%rbx
crash>
So then I tried using "mod -S <path>" using the path containing
the <module>.ko.debug files, and that works OK:
# /usr/bin/crash
crash 6.0.2-1.fc16
Copyright (C) 2002-2011 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/vmlinux
DUMPFILE: /dev/crash
CPUS: 4
DATE: Tue Jan 31 10:06:05 2012
UPTIME: 00:57:50
LOAD AVERAGE: 0.01, 0.04, 0.10
TASKS: 111
NODENAME: dell-pec5125-02.lab.bos.redhat.com
RELEASE: 3.2.2-1.fc16.x86_64
VERSION: #1 SMP Thu Jan 26 03:21:58 UTC 2012
MACHINE: x86_64 (2599 Mhz)
MEMORY: 2 GB
PID: 1445
COMMAND: "crash"
TASK: ffff880078410000 [THREAD_INFO: ffff88003e310000]
CPU: 1
STATE: TASK_RUNNING (ACTIVE)
crash> mod -S /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64
MODULE NAME SIZE OBJECT FILE
ffffffffa0002420 pata_atiixp 13197 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/pata_atiixp.ko.debug
ffffffffa000a5c0 ata_generic 12899 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/ata_generic.ko.debug
ffffffffa0012420 pata_acpi 13027 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/pata_acpi.ko.debug
ffffffffa0041580 sunrpc 235041 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/net/sunrpc/sunrpc.ko.debug
ffffffffa0055360 dcdbas 14828 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/firmware/dcdbas.ko.debug
ffffffffa005a1e0 k10temp 13119 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/hwmon/k10temp.ko.debug
ffffffffa00600a0 joydev 17412 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/input/joydev.ko.debug
ffffffffa0069140 edac_mce_amd 22882 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/edac_mce_amd.ko.debug
ffffffffa0073f20 edac_core 51619 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/edac_core.ko.debug
ffffffffa007c060 dca 14601 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/dca/dca.ko.debug
ffffffffa00834c0 amd64_edac_mod 23795 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/amd64_edac_mod.ko.debug
ffffffffa008c600 i2c_core 37955 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/i2c/i2c-core.ko.debug
ffffffffa0093cc0 i2c_piix4 13694 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/i2c/busses/i2c-piix4.ko.debug
ffffffffa0098120 sp5100_tco 13697 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/watchdog/sp5100_tco.ko.debug
ffffffffa00a7380 microcode 23240 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/arch/x86/kernel/microcode.ko.debug
ffffffffa00ba9a0 lockd 84578 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/fs/lockd/lockd.ko.debug
ffffffffa00e3000 igb 147890 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/net/ethernet/intel/igb/igb.ko.debug
ffffffffa00ed140 crash 12576 /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/char/crash.ko.debug
crash> dis -l rpc_sleep_on_priority 10
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/net/sunrpc/sched.c: 351
0xffffffffa0022650 <rpc_sleep_on_priority>: push %rbp
0xffffffffa0022651 <rpc_sleep_on_priority+1>: mov %rsp,%rbp
0xffffffffa0022654 <rpc_sleep_on_priority+4>: sub $0x20,%rsp
0xffffffffa0022658 <rpc_sleep_on_priority+8>: mov %rbx,-0x20(%rbp)
0xffffffffa002265c <rpc_sleep_on_priority+12>: mov %r12,-0x18(%rbp)
0xffffffffa0022660 <rpc_sleep_on_priority+16>: mov %r13,-0x10(%rbp)
0xffffffffa0022664 <rpc_sleep_on_priority+20>: mov %r14,-0x8(%rbp)
0xffffffffa0022668 <rpc_sleep_on_priority+24>: data32 data32 data32 xchg %ax,%ax
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/arch/x86/include/asm/bitops.h: 312
0xffffffffa002266d <rpc_sleep_on_priority+29>: mov 0x70(%rsi),%rax
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/net/sunrpc/sched.c: 351
0xffffffffa0022671 <rpc_sleep_on_priority+33>: mov %rdi,%rbx
crash>
And then finally, with "mod -S" pointing to the base "/lib/modules" path:
# /usr/bin/crash
crash 6.0.2-1.fc16
Copyright (C) 2002-2011 Red Hat, Inc.
Copyright (C) 2004, 2005, 2006 IBM Corporation
Copyright (C) 1999-2006 Hewlett-Packard Co
Copyright (C) 2005, 2006 Fujitsu Limited
Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
Copyright (C) 2005 NEC Corporation
Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
This program is free software, covered by the GNU General Public License,
and you are welcome to change it and/or distribute copies of it under
certain conditions. Enter "help copying" to see the conditions.
This program has absolutely no warranty. Enter "help warranty" for details.
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-unknown-linux-gnu"...
KERNEL: /usr/lib/debug/lib/modules/3.2.2-1.fc16.x86_64/vmlinux
DUMPFILE: /dev/crash
CPUS: 4
DATE: Tue Jan 31 10:11:35 2012
UPTIME: 01:03:20
LOAD AVERAGE: 0.24, 0.10, 0.10
TASKS: 111
NODENAME: dell-pec5125-02.lab.bos.redhat.com
RELEASE: 3.2.2-1.fc16.x86_64
VERSION: #1 SMP Thu Jan 26 03:21:58 UTC 2012
MACHINE: x86_64 (2599 Mhz)
MEMORY: 2 GB
PID: 1799
COMMAND: "crash"
TASK: ffff88007ba8c560 [THREAD_INFO: ffff88003e310000]
CPU: 3
STATE: TASK_RUNNING (ACTIVE)
crash> mod -S /lib/modules/3.2.2-1.fc16.x86_64
MODULE NAME SIZE OBJECT FILE
ffffffffa0002420 pata_atiixp 13197 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/pata_atiixp.ko
ffffffffa000a5c0 ata_generic 12899 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/ata_generic.ko
ffffffffa0012420 pata_acpi 13027 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/ata/pata_acpi.ko
ffffffffa0041580 sunrpc 235041 /lib/modules/3.2.2-1.fc16.x86_64/kernel/net/sunrpc/sunrpc.ko
ffffffffa0055360 dcdbas 14828 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/firmware/dcdbas.ko
ffffffffa005a1e0 k10temp 13119 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/hwmon/k10temp.ko
ffffffffa00600a0 joydev 17412 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/input/joydev.ko
ffffffffa0069140 edac_mce_amd 22882 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/edac_mce_amd.ko
ffffffffa0073f20 edac_core 51619 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/edac_core.ko
ffffffffa007c060 dca 14601 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/dca/dca.ko
ffffffffa00834c0 amd64_edac_mod 23795 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/edac/amd64_edac_mod.ko
ffffffffa008c600 i2c_core 37955 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/i2c/i2c-core.ko
ffffffffa0093cc0 i2c_piix4 13694 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/i2c/busses/i2c-piix4.ko
ffffffffa0098120 sp5100_tco 13697 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/watchdog/sp5100_tco.ko
ffffffffa00a7380 microcode 23240 /lib/modules/3.2.2-1.fc16.x86_64/kernel/arch/x86/kernel/microcode.ko
ffffffffa00ba9a0 lockd 84578 /lib/modules/3.2.2-1.fc16.x86_64/kernel/fs/lockd/lockd.ko
ffffffffa00e3000 igb 147890 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/net/ethernet/intel/igb/igb.ko
ffffffffa00ed140 crash 12576 /lib/modules/3.2.2-1.fc16.x86_64/kernel/drivers/char/crash.ko
crash> dis -l rpc_sleep_on_priority 10
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/net/sunrpc/sched.c: 351
0xffffffffa0022650 <rpc_sleep_on_priority>: push %rbp
0xffffffffa0022651 <rpc_sleep_on_priority+1>: mov %rsp,%rbp
0xffffffffa0022654 <rpc_sleep_on_priority+4>: sub $0x20,%rsp
0xffffffffa0022658 <rpc_sleep_on_priority+8>: mov %rbx,-0x20(%rbp)
0xffffffffa002265c <rpc_sleep_on_priority+12>: mov %r12,-0x18(%rbp)
0xffffffffa0022660 <rpc_sleep_on_priority+16>: mov %r13,-0x10(%rbp)
0xffffffffa0022664 <rpc_sleep_on_priority+20>: mov %r14,-0x8(%rbp)
0xffffffffa0022668 <rpc_sleep_on_priority+24>: data32 data32 data32 xchg %ax,%ax
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/arch/x86/include/asm/bitops.h: 312
0xffffffffa002266d <rpc_sleep_on_priority+29>: mov 0x70(%rsi),%rax
/usr/src/debug/kernel-3.2.fc16/linux-3.2.x86_64/net/sunrpc/sched.c: 351
0xffffffffa0022671 <rpc_sleep_on_priority+33>: mov %rdi,%rbx
crash>
I'll keep trying...
Dave
More information about the Crash-utility
mailing list