[dm-devel] Kernel panic with kmpathd

Yanqing_Liu at Dell.com Yanqing_Liu at Dell.com
Thu Aug 11 16:07:15 UTC 2011 

Hello,

In running IO and failover/failback testing with Dell MD3600f, we ran into a kernel panic (NULL pointer dereference) with process kmpathd. Here is the captured serial output for this.

It will usually lead to host panic or hang.

Any idea what caused this problem?


14:23:01:757  /dev/ttyS0>
14:23:01:757  /dev/ttyS0>end_request: I/O error, dev sdf, sector 2922776
14:23:01:761  /dev/ttyS0>
14:23:01:801  /dev/ttyS0>end_request: I/O error, dev sdf, sector 2919704
14:23:01:801  /dev/ttyS0>
14:23:01:825  /dev/ttyS0>end_request: I/O error, dev sdf, sector 2893056
14:23:01:829  /dev/ttyS0>
14:23:01:861  /dev/ttyS0>end_request: I/O error, dev sdf, sector 2921752
14:23:01:865  /dev/ttyS0>
14:23:02:013  /dev/ttyS0>end_request: I/O error, dev sdf, sector 2924832
14:23:02:017  /dev/ttyS0>
14:23:22:081  /dev/ttyS0>device-mapper: multipath: Failing path 65:32.
14:23:22:085  /dev/ttyS0>
14:23:43:877  /dev/ttyS0>Unable to handle kernel NULL pointer dereference at 0000000000000008 RIP:
14:23:43:885  /dev/ttyS0>
14:23:43:885  /dev/ttyS0> [<ffffffff8004d77e>] run_workqueue+0x69/0xf6
14:23:43:889  /dev/ttyS0>
14:23:43:889  /dev/ttyS0>PGD 2011cd067 PUD 1f051d067 PMD 0
14:23:43:889  /dev/ttyS0>
14:23:43:889  /dev/ttyS0>Oops: 0002 [1] SMP
14:23:43:893  /dev/ttyS0>
14:23:43:893  /dev/ttyS0>last sysfs file: /devices/pci0000:00/0000:00:07.0/0000:06:00.1/host4/rport-4:0-0/targ
14:23:43:901  /dev/ttyS0>et4:0:0/4:0:0:10/timeout
14:23:43:901  /dev/ttyS0>
14:23:43:901  /dev/ttyS0>CPU 9
14:23:43:905  /dev/ttyS0>
14:23:43:905  /dev/ttyS0>Modules linked in: autofs4 hidp nls_utf8 cifs rfcomm l2cap bluetooth lockd sunrpc be2iscsi ib_iser rdma_cm ib
14:23:43:913  /dev/ttyS0>_cm iw_cm ib_sa ib_mad ib_addr iscsi_tcp bnx2i cnic ipv6 xfrm_nalgo crypto_api uio cxgb3i iw_cxgb3 ib_core cxg
14:23:43:921  /dev/ttyS0>b3 8021q libiscsi_tcp libiscsi2 scsi_transport_iscsi2 scsi_transport_iscsi loop dm_round_robin s
14:23:43:933  /dev/ttyS0>csi_dh_rdac(U) dm_multipath scsi_dh video backlight sbs power_meter hwmon i2c_ec i2c_core dell_wmi wmi b
14:23:43:941  /dev/ttyS0>utton battery asus_acpi acpi_memhotplug ac parport_pc lp parport sr_mod cdrom tpm_tis tpm i7core_edac edac
14:23:43:949  /dev/ttyS0>_mc tpm_bios serio_raw sg(U) bnx2 pcspkr dm_raid45 dm_message dm_region_hash dm_mem_cache dm_snapsho
14:23:43:957  /dev/ttyS0>t dm_zero dm_mirror dm_log dm_mod qla2xxx scsi_transport_fc ata_piix libata shpchp megaraid_sas sd_mod s
14:23:43:969  /dev/ttyS0>csi_mod ext3 jbd uhci_hcd ohci_hcd ehci_hcd
14:23:43:973  /dev/ttyS0>
14:23:43:973  /dev/ttyS0>Pid: 4429, comm: kmpathd/9 Tainted: G      2.6.18-238.el5 #1
14:23:43:977  /dev/ttyS0>
14:23:43:977  /dev/ttyS0>RIP: 0010:[<ffffffff8004d77e>]  [<ffffffff8004d77e>] run_workqueue+0x69/0xf6
14:23:43:985  /dev/ttyS0>
14:23:43:985  /dev/ttyS0>RSP: 0018:ffff81062b4c5e40  EFLAGS: 00010006
14:23:43:989  /dev/ttyS0>
14:23:43:989  /dev/ttyS0>RAX: 0000000000000000 RBX: ffff81062e588ea0 RCX: 0000000000000000
14:23:43:993  /dev/ttyS0>
14:23:43:993  /dev/ttyS0>RDX: 0000000000000000 RSI: 0000000000000246 RDI: ffff81032f972140
14:23:44:001  /dev/ttyS0>
14:23:44:001  /dev/ttyS0>RBP: ffff81062e588ea8 R08: ffff81032f972190 R09: 000000000052ac18
14:23:44:005  /dev/ttyS0>
14:23:44:005  /dev/ttyS0>R10: 0000000000000000 R11: ffff810085c03af0 R12: ffff81032f972140
14:23:44:009  /dev/ttyS0>
14:23:44:009  /dev/ttyS0>R13: 0000000000000246 R14: 0000000000000000 R15: 0000000000000000
14:23:44:017  /dev/ttyS0>
14:23:44:017  /dev/ttyS0>FS:  0000000000000000(0000) GS:ffff81032fdd2840(0000) knlGS:0000000000000000
14:23:44:025  /dev/ttyS0>
14:23:44:025  /dev/ttyS0>CS:  0010 DS: 0018 ES: 0018 CR0: 000000008005003b
14:23:44:029  /dev/ttyS0>
14:23:44:029  /dev/ttyS0>CR2: 0000000000000008 CR3: 00000001d98a3000 CR4: 00000000000006e0
14:23:44:033  /dev/ttyS0>
14:23:44:033  /dev/ttyS0>Process kmpathd/9 (pid: 4429, threadinfo ffff81062b4c4000, task ffff81032d2300c0)
14:23:44:063  /dev/ttyS0>
14:23:44:063  /dev/ttyS0>Stack:  ffff81062b4c5e80 ffff81032f972140 ffffffff80049ff6 ffff81062b435da8
14:23:44:065  /dev/ttyS0>
14:23:44:065  /dev/ttyS0> 0000000000000282 ffff81062b435d98 ffffffff800a269c ffffffff8004a0e6
14:23:44:065  /dev/ttyS0>
14:23:44:065  /dev/ttyS0> 0000000000000000 ffff81032d2300c0 ffffffff8008e40a 0000000000100100
14:23:44:065  /dev/ttyS0>
14:23:44:065  /dev/ttyS0>Call Trace:
14:23:44:065  /dev/ttyS0>
14:23:44:065  /dev/ttyS0> [<ffffffff80049ff6>] worker_thread+0x0/0x122
14:23:44:065  /dev/ttyS0>
14:23:44:065  /dev/ttyS0> [<ffffffff800a269c>] keventd_create_kthread+0x0/0xc4
14:23:44:071  /dev/ttyS0>
14:23:44:071  /dev/ttyS0> [<ffffffff8004a0e6>] worker_thread+0xf0/0x122
14:23:44:073  /dev/ttyS0>
14:23:44:073  /dev/ttyS0> [<ffffffff8008e40a>] default_wake_function+0x0/0xe
14:23:44:077  /dev/ttyS0>
14:23:44:077  /dev/ttyS0> [<ffffffff800a269c>] keventd_create_kthread+0x0/0xc4
14:23:44:085  /dev/ttyS0>
14:23:44:085  /dev/ttyS0> [<ffffffff80032974>] kthread+0xfe/0x132
14:23:44:089  /dev/ttyS0>
14:23:44:089  /dev/ttyS0> [<ffffffff8005dfb1>] child_rip+0xa/0x11
14:23:44:089  /dev/ttyS0>
14:23:44:089  /dev/ttyS0> [<ffffffff800a269c>] keventd_create_kthread+0x0/0xc4
14:23:44:097  /dev/ttyS0>
14:23:44:097  /dev/ttyS0> [<ffffffff80032876>] kthread+0x0/0x132
14:23:44:101  /dev/ttyS0>
14:23:44:101  /dev/ttyS0> [<ffffffff8005dfa7>] child_rip+0x0/0x11
14:23:44:101  /dev/ttyS0>
14:23:44:101  /dev/ttyS0>
14:23:44:101  /dev/ttyS0>
14:23:44:101  /dev/ttyS0>
14:23:44:105  /dev/ttyS0>
14:23:44:105  /dev/ttyS0>Code: 48 89 42 08 48 89 10 48 89 6d 08 48 89 6d 00 e8 76 73 01 00
14:23:44:109  /dev/ttyS0>
14:23:44:109  /dev/ttyS0>RIP  [<ffffffff8004d77e>] run_workqueue+0x69/0xf6
14:23:44:113  /dev/ttyS0>
14:23:44:113  /dev/ttyS0> RSP <ffff81062b4c5e40>
14:23:44:117  /dev/ttyS0>

Thanks,
Yanqing
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20110811/20c29f4b/attachment.htm>

More information about the dm-devel mailing list