<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"> <head> <meta http-equiv=Content-Type content="text/html; charset=us-ascii"> <meta name=Generator content="Microsoft Word 12 (filtered medium)"> <style>  </style>  </head> <body lang=EN-US link=blue vlink=purple> <div class=Section1> Hi,<o:p></o:p> <o:p> </o:p> We are running with multipath over MD3xxx ISCSI storage (RDAC).<o:p></o:p> Our test scripts perform a series of multipathd restarts, udevtrigger and udevsettle calls, during which we logout of the storage.<o:p></o:p> Sometimes we get the following panic:<o:p></o:p> <o:p> </o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: lun5: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 MAJOR: multipathd: dm-11: devmap already registered<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 MAJOR: multipathd: dm-11: devmap already registered<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-11: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 MAJOR: multipathd: dm-11: devmap already registered<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-0: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath0: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-1: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath1: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-2: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath2: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-3: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath3: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-4: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath4: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: lun5: event checker exit<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-5: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath6: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-6: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath7: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-7: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath8: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-8: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath9: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-9: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath10: event checker started<o:p></o:p> 2011 Jan 26 22:27:01 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: dm-10: add map (uevent)<o:p></o:p> 2011 Jan 26 22:27:01 node0 NOTICE: multipathd: mpath11: event checker started<o:p></o:p> 2011 Jan 26 22:27:02 node0 INFO: kernel: sd 194:0:0:4: rdac: array MD3200i-c7-d7, ctlr 0, queueing MODE_SELECT command<o:p></o:p> 2011 Jan 26 22:27:02 node0 DEBUG: hba_loader:iscsi_logout:1130: Started<o:p></o:p> 2011 Jan 26 22:27:02 node0 DEBUG: hba_loader:iscsi_logout:1130: iscsi_luns_num=1 logout_iqn=<o:p></o:p> 2011 Jan 26 22:27:02 node0 DEBUG: hba_loader:iscsi_logout:1130: Performing logout of all iSCSI nodes<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:0: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:1: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:0: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:2: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:3: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:1: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:4: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:2: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:5: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:0: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: multipathd: sdb: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:3: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:1: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:4: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:2: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:3: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:5: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:4: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:6: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:5: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:6: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:7: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:7: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:8: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:8: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:9: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:9: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:6: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:10: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:7: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:8: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 196:0:0:11: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:9: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:10: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 194:0:0:11: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:0: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 195:0:0:10: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:1: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:2: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:3: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:4: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:5: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:6: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:7: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:8: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:9: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: kernel: scsi 193:0:0:10: rdac Dettached<o:p></o:p> 2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:02 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:02 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:02 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:02 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: multipathd: mpath8: failed in domap for removal of path sdb<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: multipathd: sdd: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: multipathd: mpath9: failed in domap for removal of path sdd<o:p></o:p> 2011 Jan 26 22:27:02 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:02 node0 NOTICE: multipathd: sdh: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath8: failed in domap for removal of path sdh<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdf: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath10: failed in domap for removal of path sdf<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdo: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath9: failed in domap for removal of path sdo<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdj: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: scsi 194:0:0:8: rejecting I/O to dead device<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 65:112.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: scsi 195:0:0:8: rejecting I/O to dead device<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 65:160.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: multipath: Cannot failover device because scsi_dh_rdac was not loaded.<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 66:208.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: multipath: Cannot failover device because scsi_dh_rdac was not loaded.<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: multipath: Failing path 66:224.<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: printk: 46 messages suppressed.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 1<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 2<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 3<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 70778879<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 70778879<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 0<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: Buffer I/O error on device dm-0, logical block 3<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath11: failed in domap for removal of path sdj<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdl: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath3: failed in domap for removal of path sdl<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 65:160: mark as failed<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 3<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 65:112: mark as failed<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 2<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 66:208: mark as failed<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 1<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: 66:224: mark as failed<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: mpath0: remaining active paths: 0<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdt: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath10: failed in domap for removal of path sdt<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdn: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath4: failed in domap for removal of path sdn<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdy: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath11: failed in domap for removal of path sdy<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdc: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:7: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath8: failed in domap for removal of path sdc<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sde: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:8: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath9: failed in domap for removal of path sde<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdaf: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath3: failed in domap for removal of path sdaf<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdg: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:9: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath10: failed in domap for removal of path sdg<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdah: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath4: failed in domap for removal of path sdah<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdk: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:10: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath11: failed in domap for removal of path sdk<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:3: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdm: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath3: failed in domap for removal of path sdm<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdan: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath6: failed in domap for removal of path sdan<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdq: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:4: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath4: failed in domap for removal of path sdq<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:6: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:6: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdap: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath7: failed in domap for removal of path sdap<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sds: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:5: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: mpath6: failed in domap for removal of path sds<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: multipathd: uevent trigger error<o:p></o:p> 2011 Jan 26 22:27:03 node0 NOTICE: multipathd: sdw: remove path (uevent)<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 node0 MAJOR: kernel: device-mapper: table: 253:6: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 node0 WARNING: kernel: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 node0 INFO: kernel: device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for failover/failback and device management.<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ----------- [cut here ] --------- [please bite here ] ---------<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Kernel BUG at drivers/scsi/device_handler/scsi_dh_rdac.c:252<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: invalid opcode: 0000 [1] <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: SMP <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: last sysfs file: /class/iscsi_session/session9/ifacename<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: CPU 2 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Modules linked in:<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: iscsi_tcp(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: libiscsi_tcp(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: libiscsi2(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_transport_iscsi2(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_transport_iscsi(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipmi_si(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_dh_rdac(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_rdac(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_queue_depth(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: netconsole(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_round_robin(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptctl(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dell_rbu(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: exa_ioctls(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: nfs(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: lockd(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: nfs_acl(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: xt_tcpudp(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: iptable_filter(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ip_tables(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipt_SYSRQ(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: x_tables(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sunrpc(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipmi_devintf(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipmi_msghandler(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: bonding1(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: bonding(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ipv6(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: xfrm_nalgo(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: crypto_api(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_mirror(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_log(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_multipath(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_dh(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: dm_mod(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: video(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: hwmon(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: backlight(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sbs(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: i2c_ec(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: i2c_core(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: button(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: battery(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: asus_acpi(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ac(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: joydev(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sr_mod(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: cdrom(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: bnx2(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sg(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: pcspkr(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ata_piix(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: libata(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptsas(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptscsih(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: mptbase(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_transport_sas(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: sd_mod(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: scsi_mod(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ext3(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: jbd(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: uhci_hcd(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ohci_hcd(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ehci_hcd(U)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Pid: 12912, comm: kmpath_rdacd Tainted: G 2.6.18-164sys #1<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RIP: 0010:[<ffffffff8838dc4f>] <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8838dc4f>] :scsi_dh_rdac:send_mode_select+0x2cf/0x4b0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: device-mapper: table: 253:6: multipath: unknown path selector type<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: device-mapper: ioctl: error adding target to table<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RSP: 0018:ffff810086959d80 EFLAGS: 00010246<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RAX: 0000000000000000 RBX: 00000000fffffffb RCX: 0000000000000000<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RDX: 0000000000000800 RSI: 0000000000000286 RDI: ffff810055e2d800<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RBP: ffff81005a647216 R08: ffffffff80370b58 R09: 0000000000000001<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: R10: 0000000000000046 R11: ffffffff8004c200 R12: 0000000000000000<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: R13: 0000000000000000 R14: ffff8102352bc000 R15: ffff8102352bc062<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: FS: 0000000000000000(0000) GS:ffff810107eda340(0000) knlGS:0000000000000000<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: CS: 0010 DS: 0018 ES: 0018 CR0: 000000008005003b<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: CR2: 00002ae8ff6760a0 CR3: 000000023487c000 CR4: 00000000000006e0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Process kmpath_rdacd (pid: 12912, threadinfo ffff810086958000, task ffff810092a54140)<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Stack: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0000000010008040<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81005b23a5c0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810086959e70<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810086959dd0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810001004498<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff810055e2d800<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81005a647208<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81004d3226f0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0000000380438420<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0000000000000001<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81023a498d80<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: ffff81023a498d80<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Call Trace:<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8838d980>] :scsi_dh_rdac:send_mode_select+0x0/0x4b0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800561a3>] run_workqueue+0xb3/0x110<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80052180>] worker_thread+0x0/0x150<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800b33b0>] keventd_create_kthread+0x0/0xa0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80052291>] worker_thread+0x111/0x150<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8009ca00>] default_wake_function+0x0/0x10<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80052180>] worker_thread+0x0/0x150<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800373c9>] kthread+0xd9/0x120<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80068fb1>] child_rip+0xa/0x11<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800b33b0>] keventd_create_kthread+0x0/0xa0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff800372f0>] kthread+0x0/0x120<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff80068fa7>] child_rip+0x0/0x11<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: Code: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0f <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0b <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 68 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 98 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: e1 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 38 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 88 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: c2 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: fc <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 0f <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 1f <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 80 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 00 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 48 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 8d <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: 54 <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RIP <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: [<ffffffff8838dc4f>] :scsi_dh_rdac:send_mode_select+0x2cf/0x4b0<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: RSP <ffff810086959d80><o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 EMERG: Kernel panic - not syncing: Fatal exception<o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 NOTICE: <o:p></o:p> 2011 Jan 26 22:27:03 172.19.59.130 EMERG: Rebooting in 1 seconds..<o:p></o:p> <o:p> </o:p> <o:p> </o:p> This panic comes from get_rdac_data, which is called from within the send_mode_select code.<o:p></o:p> My analysis picked up the following problem:<o:p></o:p> The RDAC detach code nullifies the scsi_dh_data before flushing the mode select workqueue, and as result a pending send_mode_select will BUG_ON in get_rdac_data.<o:p></o:p> The following patch (tested over RHEL54) flushes the workqueue before setting scsi_dh_data to NULL:<o:p></o:p> <o:p> </o:p> diff -r -U 2 a/drivers/scsi/device_handler/scsi_dh_rdac.c b/drivers/scsi/device_handler/scsi_dh_rdac.c<o:p></o:p> --- a/drivers/scsi/device_handler/scsi_dh_rdac.c 2011-01-30 11:02:31.271426000 +0200<o:p></o:p> +++ b/drivers/scsi/device_handler/scsi_dh_rdac.c 2011-01-30 11:02:31.327069000 +0200<o:p></o:p> @@ -853,8 +853,22 @@<o:p></o:p> spin_lock_irqsave(sdev->request_queue->queue_lock, flags);<o:p></o:p> scsi_dh_data = retrieve_scsi_dh_data(sdev);<o:p></o:p> - store_scsi_dh_data(sdev, NULL);<o:p></o:p> spin_unlock_irqrestore(sdev->request_queue->queue_lock, flags);<o:p></o:p> <o:p> </o:p> h = (struct rdac_dh_data *) scsi_dh_data->buf;<o:p></o:p> + if (h->ctlr) {<o:p></o:p> + int flush;<o:p></o:p> +<o:p></o:p> + spin_lock(&h->ctlr->ms_lock);<o:p></o:p> + flush = (h->ctlr->ms_sdev == sdev);<o:p></o:p> + spin_unlock(&h->ctlr->ms_lock);<o:p></o:p> +<o:p></o:p> + if (flush)<o:p></o:p> + flush_workqueue(kmpath_rdacd);<o:p></o:p> + }<o:p></o:p> +<o:p></o:p> + spin_lock_irqsave(sdev->request_queue->queue_lock, flags);<o:p></o:p> + store_scsi_dh_data(sdev, NULL);<o:p></o:p> + spin_unlock_irqrestore(sdev->request_queue->queue_lock, flags);<o:p></o:p> +<o:p></o:p> if (h->ctlr)<o:p></o:p> kref_put(&h->ctlr->kref, release_controller);<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <o:p> </o:p> The patch ensures that we run flush only when detaching the device the “owns” the send_mode_select (ms_sdev).<o:p></o:p> <o:p> </o:p> Menny<o:p></o:p> <o:p> </o:p> <o:p> </o:p> <o:p> </o:p> </div> </body> </html>