[dm-devel] Multipath ID not equal to LUN scsi ID

Wed Jul 12 02:14:55 UTC 2017

Thanks Steffen.

Our management tool(Openstack) will do as the clean as the document(https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/removing_devices.html)
in normal situation, but Openstack do have bugs which will lead to paths of unmapped LUN not been removed. This will cause data crush if the original paths are added back. 
Although Openstack should not leave the paths, multipath should found out different LUNs by different ID_SERIAL. Any FC administor may forget to delete the paths before adding a new LUN with old LUN ID.

Flush the path and rescan scsi bus and then do a multipath -v2 could make the new path currect. I am thinking whether we could add similar steps in multipath-tool when multipath detected a offlined path become
online(no udev event about new arrived LUN in this situation).  Not an expert of the multipath code, do you think this is reasonable?

===============================================
Since Linux does not really automatically handle target volume
remapping, you might be better of with explicitly tearing down the old
volume in Linux before any unmap/remap on the target. This is already
tricky enough to get right.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/removing_devices.html

Resizing of the same volume on the target is a special case which should
work, but even that one includes a rescan of the affected scsi devices
(paths) as pre-req.

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/online-iscsi-resizing.html#resizing-fc-logical-units
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/online-iscsi-resizing.html#idm8373264
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/DM_Multipath/online_device_resize.html

NB: None of the URLs provided are specific to a distro or device driver.
Those were just example documentation references I could find quickly.

On 07/11/2017 11:07 AM, Steffen Maier wrote:
> On 07/11/2017 05:49 AM, liuqing at huayun.com wrote:
>> Once a new arrived LUN mapped we will do rescan by "echo '- - -' >
>> /sys/class/scsi_host/host2/scan".
>
> I was referring to a rescan of the scsi device,
>
> https://www.ibm.com/support/knowledgecenter/linuxonibm/com.ibm.linux.z.lhdd/lhdd_t_fcp_wrk_rescan.html
>
>
> not a scan of the Scsi_Host.
> The latter likely keeps the old scsi device with its old properties
> being oblivious to the volume remapping on the storage target; it's
> about discovering new scsi devices previously not being known to Linux.
>
>> After rescan only the scsi_id tool give the right serial id, udevadm
>> still got the prvious one.
>> I have monitor the udev event by udevadm monitor while mapping a new
>> LUN to the host, who will reuse the original path. No add event is
>> triggerred, only dm-X emits a change event.
>> But if the original path is deleted(removed) then add event will be
>> triggerred.
>>
>> Flush the old WWID could make the WWID correct but the size is still
>> incorrect as following. And 36005076300810eadf800000000000155 is
>> actually 5GB.
>
> likely same reason as above
>
>> I built the tool using latest code and tried both attribute_id and
>> getuid_callout. The issue exist in both configuration.
>>
>>
>>> Dear list,
>>> We have a FC storage and using multipathd to manager the FC paths.
>>> I've met an issue in this environment. The following is how to
>>> recreate the issue.
>>>
>>> =======
>>> 1. Map a LUN to host with LUN ID 0,
>>> 2. rescan fc_host, a new path will be found by multipath.
>>> 3. Unmap LUN 0.  path will failed as following.
>>> [root at localhost sys]# multipath -ll
>>> Jul 10 18:41:50 | sdp: couldn't get asymmetric access state
>>> Jul 10 18:41:50 | sdq: couldn't get asymmetric access state
>>> 36005076300810eadf800000000000156 dm-3 IBM,2145
>>> size=8.0G features='2 queue_if_no_path retain_attached_hw_handler'
>>> hwhandler='1 alua' wp=rw
>>> |-+- policy='service-time 0' prio=0 status=enabled
>>> | `- 2:0:0:0 sdp 8:240 failed faulty running
>>> `-+- policy='service-time 0' prio=0 status=enabled
>>>     `- 2:0:1:0 sdq 65:0  failed faulty running
>>> 4. Map another LUN which have different ID_SERIAL but with the same
>>> LUN ID(0).
>> Did you "rescan" the SCSI device via sysfs to let Linux know that this
>> is now in fact a different device?
>> AFAIK, Linux decodes SCSI sense data for LUNs remapped on the storage
>> target and emits a udev event, but I'm not aware of any default udev
>> rule that would actually react. The kernel itself does not react other
>> than creating the uevent.
>
>

--
Mit freundlichen Gr??en / Kind regards
Steffen Maier

Linux on z Systems Development

IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschaeftsfuehrung: Dirk Wittkopp
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294

liuqing at huayun.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/dm-devel/attachments/20170712/beebcecc/attachment.htm>