Dave,<br>Thanks for pointing out this feature of crash.<br>It will be of great help.<br><font color="#888888"><br>Dheeraj</font><br><br><div class="gmail_quote">On Tue, Mar 4, 2008 at 2:49 AM, Dave Anderson <<a href="mailto:anderson@redhat.com">anderson@redhat.com</a>> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div><div></div><div class="Wj3C7c"><br>
Dheeraj Sangamkar wrote:<br>
><br>
> Hi,<br>
> Is there a way to print type information about types defined in a loadable<br>
> kernel module? Is there a way to add to the type information that crash gets<br>
> from the vmlinux file?<br>
><br>
> This would ease debugging crashed module code be helping me print structures<br>
> defined in modules.<br>
><br>
> Dheeraj<br>
<br>
</div></div>Well, that's what the "mod" command does, i.e., load the debuginfo data<br>
of kernel modules into the embedded gdb module of the crash session.<br>
<br>
Take the "ext3" module for example. It declares this data structure in<br>
the file "fs/ext3/super.c":<br>
<br>
static struct super_operations ext3_sops = {<br>
.alloc_inode = ext3_alloc_inode,<br>
.destroy_inode = ext3_destroy_inode,<br>
.read_inode = ext3_read_inode,<br>
.write_inode = ext3_write_inode,<br>
.dirty_inode = ext3_dirty_inode,<br>
.delete_inode = ext3_delete_inode,<br>
.put_super = ext3_put_super,<br>
.write_super = ext3_write_super,<br>
.sync_fs = ext3_sync_fs,<br>
.write_super_lockfs = ext3_write_super_lockfs,<br>
.unlockfs = ext3_unlockfs,<br>
.statfs = ext3_statfs,<br>
.remount_fs = ext3_remount,<br>
.clear_inode = ext3_clear_inode,<br>
.show_options = ext3_show_options,<br>
#ifdef CONFIG_QUOTA<br>
.quota_read = ext3_quota_read,<br>
.quota_write = ext3_quota_write,<br>
#endif<br>
};<br>
<br>
or for example, this data structure type is defined in<br>
"fs/ext3/xattr.h":<br>
<br>
struct ext3_xattr_header {<br>
__le32 h_magic; /* magic number for identification */<br>
__le32 h_refcount; /* reference count */<br>
__le32 h_blocks; /* number of disk blocks used */<br>
__le32 h_hash; /* hash value of all attributes */<br>
__u32 h_reserved[4]; /* zero right now */<br>
};<br>
<br>
Without the ext3 module's debuginfo data, crash (and gdb) doesn't know<br>
what they are, so if I try to print the "ext3_sops" symbolically,<br>
this happens:<br>
<br>
crash> p ext3_sops<br>
p: gdb request failed: p ext3_sops<br>
crash> whatis ext3_sops<br>
whatis: gdb request failed: whatis ext3_sops<br>
crash><br>
<br>
And if I want to see what an ext3_xattr_header structure is defined<br>
as:<br>
<br>
crash> struct ext3_xattr_header<br>
struct: invalid data structure reference: ext3_xattr_header<br>
crash><br>
<br>
But if the module's debuginfo data is loaded:<br>
<br>
crash> mod -s ext3<br>
MODULE NAME SIZE OBJECT FILE<br>
e08c4f80 ext3 123337<br>
/lib/modules/2.6.18-53.el5/kernel/fs/ext3/ext3.ko<br>
crash><br>
<br>
(or use "mod -S" to load the debuginfo data of all modules),<br>
then the same commands that failed above work like so:<br>
<br>
crash> p ext3_sops<br>
ext3_sops = $4 = {<br>
alloc_inode = 0xe08b1c54 <ext3_alloc_inode>,<br>
destroy_inode = 0xe08b2e4a <ext3_destroy_inode>,<br>
read_inode = 0xe08ae0c9 <ext3_read_inode>,<br>
dirty_inode = 0xe08ae063 <ext3_dirty_inode>,<br>
write_inode = 0xe08ad696 <ext3_write_inode>,<br>
put_inode = 0,<br>
drop_inode = 0,<br>
delete_inode = 0xe08ad7af <ext3_delete_inode>,<br>
put_super = 0xe08b2f6d <ext3_put_super>,<br>
write_super = 0xe08b1c37 <ext3_write_super>,<br>
sync_fs = 0xe08b291d <ext3_sync_fs>,<br>
write_super_lockfs = 0xe08b295f <ext3_write_super_lockfs>,<br>
unlockfs = 0xe08b29ae <ext3_unlockfs>,<br>
statfs = 0xe08b2c55 <ext3_statfs>,<br>
remount_fs = 0xe08b2a21 <ext3_remount>,<br>
clear_inode = 0xe08b2e5a <ext3_clear_inode>,<br>
umount_begin = 0,<br>
show_options = 0xe08b2835 <ext3_show_options>,<br>
show_stats = 0,<br>
quota_read = 0xe08b449b <ext3_quota_read>,<br>
quota_write = 0xe08b45f5 <ext3_quota_write><br>
}<br>
crash><br>
<br>
crash> ext3_xattr_header<br>
struct ext3_xattr_header {<br>
__le32 h_magic;<br>
__le32 h_refcount;<br>
__le32 h_blocks;<br>
__le32 h_hash;<br>
__u32 h_reserved[4];<br>
}<br>
SIZE: 32<br>
crash><br>
<br>
The example above presumes the Red Hat split debuginfo module<br>
format, where the "mod" command shows that the ext3 module<br>
exists in:<br>
<br>
/lib/modules/2.6.18-53.el5/kernel/fs/ext3/ext3.ko,<br>
<br>
but that ext3.ko module contains a link to its associated debuginfo<br>
part, which is actually located in:<br>
<br>
/usr/lib/debug/lib/modules/2.6.18-53.el5/kernel/fs/ext3/ext3.ko.debug<br>
<br>
If your module.ko file is not split into two, and has been built<br>
with -g, then you can load its debuginfo data crash like so:<br>
<br>
crash> mod -s yourmodule yourmodule.ko<br>
<font color="#888888"><br>
Dave<br>
<br>
</font></blockquote></div><br>