[Crash-utility] crash mod -S problem
Cliff Wickman
cpw at sgi.com
Thu Jan 29 17:33:07 UTC 2009
On Thu, Jan 29, 2009 at 11:23:49AM -0500, Dave Anderson wrote:
>
> ----- "Cliff Wickman" <cpw at sgi.com> wrote:
>
> > Hi Dave, and company,
> >
> > I have a question. Perhaps this is a known issue -- should I be looking
> > in an archive somewhere?
> >
> > The crash command "mod -S" is messing up the address list.
> >
> > This is crash-4.0-7.6, running on ia64.
> >
> > Note how the value of module symbol xfs_ioerror_alert has changed
> > after the mod -S.
> >
> > Is this a crash problem? Or my problem? (there are several ?'s in
> > the mod -S output, including on the xfs module)
>
> Good question -- I don't know.
>
> It appears that the determination of the module's text starting location
> is being changed by the "mod -S". That can happen when the symbols.c
> function store_load_module_symbols() calls calculate_load_order_v2(),
> which can change the text start location that was initialized during
> the crash session initialization by store_module_symbols_v2().
>
> The question is which text start address value is correct.
>
> Try this:
>
> $ crash -s
> crash> help -s | grep mod_text_start > before
> crash> mod -S
> ... [ snip ] ...
> crash> help -s | grep mod_text_start > after
> crash> q
>
> $ diff before after
It changed:
$ diff before after
1,3c1,3
< mod_text_start: a0000002093f0000 (0)
< mod_text_start: a000000209410000 (0)
< mod_text_start: a000000209430000 (0)
---
> mod_text_start: a0000002093f49e0 (49e0)
> mod_text_start: a000000209412040 (2040)
> mod_text_start: a000000209433968 (3968)
5,7c5,7
< mod_text_start: a000000209470000 (0)
< mod_text_start: a0000002094b0000 (0)
< mod_text_start: a000000209510000 (0)
---
> mod_text_start: a000000209483c68 (13c68)
> mod_text_start: a0000002094d31a8 (231a8)
> mod_text_start: a00000020951c4d8 (c4d8)
[snip]
> Typically it won't change, and the before and after files are identical.
>
> But on your system it would appear that many (if not all) of the modules are
> changing their mod_text_start values upon running "mod -S".
>
> You can also determine which value is correct by doing a "dis <function>"
> command before and after the "mod -S".
Yes, we were doing a "dis" when we noticed the problem. The addresses
are correct before the mod -S.
So you suspect the problem is in calculate_load_order_v2() ?
-Cliff
> > crash> sym xfs_ioerror_alert
> > a000000209955f80 (t) xfs_ioerror_alert
> >
> > crash> mod -S
> > MODULE NAME SIZE OBJECT FILE
> > a0000002093fc088 scsi_transport_spi 50672
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/scsi_transport_spi.ko
> > mod: ioc4: last symbol is not _MODULE_END_ioc4?
> > a000000209412fa8 ioc4 13056
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/misc/ioc4.ko
> > mod: scsi_tgt: last symbol is not _MODULE_END_scsi_tgt?
> > a000000209435cd0 scsi_tgt 24640
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/scsi_tgt.ko
> > a000000209453610 sgiioc4 14656
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/ide/pci/sgiioc4.ko
> > mod: mptbase: last symbol is not _MODULE_END_mptbase?
> > a000000209497298 mptbase 166540
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/message/fusion/mptbase.ko
> > a0000002094f8008 scsi_mod 299024
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/scsi_mod.ko
> > a000000209547038 ide_core 229280
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/ide/ide-core.ko
> > a000000209583da0 qla1280 148256
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/qla1280.ko
> > mod: scsi_transport_fc: last symbol is not
> > _MODULE_END_scsi_transport_fc?
> > a0000002095b51d0 scsi_transport_fc 88156
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/scsi_transport_fc.ko
> > a0000002095d7bd8 mptfc 32776
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/message/fusion/mptfc.ko
> > mod: mptscsih: last symbol is not _MODULE_END_mptscsih?
> > a000000209610618 mptscsih 68816
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/message/fusion/mptscsih.ko
> > a000000209631208 hwmon 5304
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/hwmon/hwmon.ko
> > mod: thermal_sys: last symbol is not _MODULE_END_thermal_sys?
> > a000000209655c90 thermal_sys 24656
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/thermal/thermal_sys.ko
> > a000000209679b40 thermal 41144
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/acpi/thermal.ko
> > a0000002096b4340 processor 85008
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/acpi/processor.ko
> > a0000002096d23c8 fan 9960
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/acpi/fan.ko
> > a000000209718490 mptspi 34936
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/message/fusion/mptspi.ko
> > a000000209730e00 ide_generic 4264
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/ide/ide-generic.ko
> > a0000002097506f0 crc_t10dif 2336
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/lib/crc-t10dif.ko
> > a000000209773430 button 14312
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/acpi/button.ko
> > a0000002097b1380 sd_mod 72432
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/sd_mod.ko
> > a0000002097e12a0 sg 71976
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/sg.ko
> > a0000002098124b8 cdrom 77752
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/cdrom/cdrom.ko
> > mod: cannot find or load object file for ide_cd_mod module
> > mod: libphy: last symbol is not _MODULE_END_libphy?
> > a000000209859928 libphy 40168
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/net/phy/libphy.ko
> > mod: xfs: last symbol is not _MODULE_END_xfs?
> > a0000002099abaf0 xfs 1233888
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/xfs/xfs.ko
> > a0000002099ed130 mptctl 188064
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/message/fusion/mptctl.ko
> > a000000209a04690 mmtimer 18836
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/char/mmtimer.ko
> > a000000209a53a68 kdbm_pg 16632
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/kdb/modules/kdbm_pg.ko
> > a000000209a70a48 kdbm_sched 3296
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/kdb/modules/kdbm_sched.ko
> > a000000209a91b28 kdbm_task 40608
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/kdb/modules/kdbm_task.ko
> > a000000209b1fba0 qla2xxx 463704
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/scsi/qla2xxx/qla2xxx.ko
> > a000000209b850c0 tg3 284796
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/net/tg3.ko
> > mod: dm_mod: last symbol is not _MODULE_END_dm_mod?
> > a000000209ff25f8 dm_mod 143048
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/md/dm-mod.ko
> > a00000020a016518 kdbm_vm 28112
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/kdb/modules/kdbm_vm.ko
> > a00000020a031f20 mspec 16932
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/char/mspec.ko
> > a00000020a05e290 numatools 59232
> > /lib/modules/2.6.27.8-1-sgidev/up
> > dates/numatools.ko
> > a00000020a073f38 xp 17216
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/misc/sgi-xp/xp.ko
> > a00000020a0b4c08 gru 87288
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/misc/sgi-gru/gru.ko
> > a00000020a0d38e0 sgi_gpl_lib 15272
> > /lib/modules/2.6.27.8-1-sgidev/we
> > ak-updates/extra/gpl_lib/sgi-gpl-lib.ko
> > a00000020a115c58 myri10ge 91048
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/net/myri10ge/myri10ge.ko
> > mod: loop: last symbol is not _MODULE_END_loop?
> > a00000020a138478 loop 34788
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/block/loop.ko
> > mod: cannot find or load object file for nls_iso8859_1 module
> > a00000020a18ac80 sgi_os_lib 111192
> > /lib/modules/2.6.27.8-1-sgidev/we
> > ak-updates/extra/os_lib/sgi-os-lib.ko
> > mod: crypto_blkcipher: last symbol is not
> > _MODULE_END_crypto_blkcipher?
> > a00000020a1a8c00 crypto_blkcipher 36848
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/crypto/crypto_blkcipher.ko
> > a00000020a1c6fd0 dm_crypt 29784
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/drivers/md/dm-crypt.ko
> > a00000020a1e4cd8 binfmt_misc 20444
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/binfmt_misc.ko
> > a00000020a25d758 xpmem 122536
> > /lib/modules/2.6.27.8-1-sgidev/up
> > dates/xpmem.ko
> > a00000020a28d518 fuse 121352
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/fuse/fuse.ko
> > mod: ipv6: last symbol is not _MODULE_END_ipv6?
> > a00000020a358030 ipv6 627656
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/net/ipv6/ipv6.ko
> > mod: sgi_xvm: last symbol is not _MODULE_END_sgi_xvm?
> > a00000020a41a520 sgi_xvm 704400
> > /lib/modules/2.6.27.8-1-sgidev/we
> > ak-updates/extra/xvm/sgi-xvm.ko
> > a00000020a46ba78 autofs4 48656
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/autofs4/autofs4.ko
> > a00000020a482090 exportfs 8944
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/exportfs/exportfs.ko
> > mod: auth_rpcgss: last symbol is not _MODULE_END_auth_rpcgss?
> > a00000020a4b3ee0 auth_rpcgss 82696
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/net/sunrpc/auth_gss/auth_rpcgss.ko
> > mod: nfs_acl: last symbol is not _MODULE_END_nfs_acl?
> > a00000020a4d17b8 nfs_acl 6648
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/nfs_common/nfs_acl.ko
> > a00000020a5b1e88 sunrpc 472176
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/net/sunrpc/sunrpc.ko
> > mod: lockd: last symbol is not _MODULE_END_lockd?
> > a00000020a5f1e00 lockd 141416
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/lockd/lockd.ko
> > a00000020a6aaa98 nfs 641664
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/nfs/nfs.ko
> > a00000020a73ea30 nfsd 523168
> > /lib/modules/2.6.27.8-1-sgidev/ke
> > rnel/fs/nfsd/nfsd.ko
> >
> > crash> sym xfs_ioerror_alert
> > a000000209992d18 (T) xfs_ioerror_alert
> >
> > Thanks.
> > -Cliff
> >
> > --
> > Crash-utility mailing list
> > Crash-utility at redhat.com
> > https://www.redhat.com/mailman/listinfo/crash-utility
>
> --
> Crash-utility mailing list
> Crash-utility at redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
--
Cliff Wickman
Silicon Graphics, Inc.
cpw at sgi.com
(651) 683-3824
More information about the Crash-utility
mailing list