[dm-devel] different LUN numbers under the same dm device

[Benjamin Marzinski]

On Wed, Jun 06, 2012 at 01:59:02PM -0700, Brian Bunker wrote:
> Mike,
> 
> The devices for LUN 12 are failed and correspond to LUN's not currently shared to the initiator at all. They were at one point and were likely used by dm-11 for its underlying paths. The inquiry data of those LUN's when the problem happened was like this:
> 
> [root at r13init32 ~]# sg_inq /dev/sde
> standard INQUIRY: [qualifier indicates no connected LU]
>   PQual=1  Device_type=31  RMB=0  version=0x06  [SPC-4]
>   [AERC=0]  [TrmTsk=0]  NormACA=0  HiSUP=0  Resp_data_format=2
>   SCCS=0  ACC=0  TPGS=0  3PC=0  Protect=0  BQue=0
>   EncServ=0  MultiP=1 (VS=0)  [MChngr=0]  [ACKREQQ=0]  Addr16=0
>   [RelAdr=0]  WBus16=0  Sync=0  Linked=0  [TranDis=0]  CmdQue=1
>   [SPI: Clocking=0x0  QAS=0  IUS=0]
>     length=96 (0x60)   Peripheral device type: no physical device on this lu
>  Vendor identification: PURE    
>  Product identification: FlashArray      
>  Product revision level: 100 
> 
> There is no NAA number, page code 0x83 or LUN serial number available, page code 0x80 since there is no LUN 12 attached as a disk device at the time multipath -ll was run. Different LUN's from our array would ever have the same NAA value, what I think you are calling UUID. 
> 
> The sequence is something like share a LUN from the array with two paths to the initiator, a dm device gets created presumably like this at first (except that the status would be active and ready and not failed and faulty:
> 
>   3624a93700a14254d729923840001000b dm-11 PURE,FlashArray
>   size=500G features='0' hwhandler='0' wp=rw
>    `-+- policy='round-robin 0' prio=1 status=active
>    |- 1:0:0:12 sde  8:64   failed faulty running
>    |- 0:0:0:12 sdd  8:48   failed faulty running
> 
> Then that LUN 12 is taken away from the initiator and the dm device dm-11 is reused later by LUN 10 when it is shared to the initiator, but the LUN 12 devices still remain as part of the dm device. Then I would expect:
> 
>  3624a93700a14254d729923840001000b dm-11 PURE,FlashArray
>  size=500G features='0' hwhandler='0' wp=rw
>  `-+- policy='round-robin 0' prio=1 status=active
>    |- 0:0:0:10 sdar 66:176 active ready  running
>    !- 1:0:0:10 sdba 67:64  active ready  running
> 
> Thanks,
> Brian

So, did sde and sdd (the paths for the multipath device for LUN 12)
originally have the wwid of 3624a93700a14254d729923840001000b.  Do
sdar and sdba actually have the wwid of
3624a93700a14254d729923840001000b.  You can check this by running

# scsi_id --whitelisted --device=<devname>

for example

# scsi_id --whitelisted --device=/dev/sde

If all of these scsi devices return the same value, then multipath
has no way of knowing that they don't belong together. If not, I'd
like to know which devices don't really have that UUID.

-Ben

> 
> On Jun 6, 2012, at 1:35 PM, Mike Snitzer wrote:
> 
> > On Wed, Jun 06 2012 at  3:27pm -0400,
> > Brian Bunker <brian at purestorage.com> wrote:
> > 
> >> Our company produces a multiple port Fibre Channel storage array. We
> >> are continually plagued by this problem. We get a dm device which
> >> combines paths for different LUN's. We would like to understand why
> >> this is happening. Wouldn't this problem almost certainly lead to a
> >> data corruption?
> >> 
> >> Thanks,
> >> Brian
> >> 
> >> 3624a93700a14254d729923840001000b dm-11 PURE,FlashArray
> >> size=500G features='0' hwhandler='0' wp=rw
> >> `-+- policy='round-robin 0' prio=1 status=active
> >>  |- 1:0:0:12 sde  8:64   failed faulty running
> >>  |- 0:0:0:12 sdd  8:48   failed faulty running
> >>  |- 0:0:0:10 sdar 66:176 active ready  running
> >>  `- 1:0:0:10 sdba 67:64  active ready  running
> >> 
> >> Of the 4 paths to dm-11, we can see two paths are for LUN 10 and the
> >> other two are for LUN 12. We have 24 other dm devices which have only
> >> the expected 2 paths.
> > 
> > Multipath considers all LUNs with the same UUID to be the same LUN.
> > 
> > So you should first try to understand why all of these paths were held
> > to have the same UUID (3624a93700a14254d729923840001000b).
> 
> Brian Bunker
> brian at purestorage.com
> 
> 
> 
> 
> --
> dm-devel mailing list
> dm-devel at redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel