[dm-devel] Status of resizing/growing dm-multipath devices on the fly

[Sharif Nassar]

On Wed, 6 Aug 2008, Pasi Kärkkäinen wrote:

> Hello!
> 
> Is it possible to resize (grow) dm-multipath devices on the fly nowadays? 
> 
> I googled and found some discussions about the subject, but the conclusion
> seemed to be it's not possible.. that was a while ago, so I was wondering if
> this has been fixed/implemented.. 
> 
> Using LVM and adding another new LUN/PV is not an option always.. it's a lot
> easier to manage the whole thing if it's possible to resize/grow existing
> LUNs on the fly.


Sure it is, but it's not pretty.  Don't believe the naysayers.
We seem to have a working method for resizing multipath luns from our 
NetApp filers.  It works on CentOS 4 & 5, for us, and our NetApps.
(2 HBA, times 2 paths on each makes for four SCSI LUN to resize)

The poorly formatted, unpolished, YMMV, no warranty recipe is:

1. Record the state of the dm tables

dmsetup table resize_test | tee orig current new final

2. Split one of the paths

multipath -l
resize_test (360a98000486e5452576f4336486d4c74) dm-2 NETAPP,LUN
[size=1.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
  \_ 5:0:0:0 sdb 8:16  [active][undef]
  \_ 6:0:0:0 sdc 8:32  [active][undef]
\_ round-robin 0 [prio=0][enabled]
  \_ 6:0:1:0 sdd 8:48  [active][undef]
  \_ 5:0:1:0 sde 8:64  [active][undef]

# Choose a pair: sdb+sdc or sdd+sde. For this, let's choose sdd+sde

# Update current
vi current
0 2097152 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000 
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000

# Change this to:
0 2097152 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:16 4000 
8:32 4000

# Note this is what happened:
# Remove the second pair beginning with the second "round-robin". We know 
it's sdd+sde based on the major/minor pairs.
# Reduce the number of pairs after "queue_if_no_path" from 2 (0 2 1) to 1 
(0 1 1)

multipathd -k
> del path sdd
> del path sde

# Reload the dm table
dmsetup suspend resize_test current; dmsetup reload resize_test current; 
dmsetup resume resize_test

# Verify you now just have one set of paths
multipath -l
resize_test (360a98000486e5452576f4336486d4c74) dm-2 NETAPP,LUN
[size=1.0G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][active]
  \_ 5:0:0:0 sdb 8:16  [active][undef]
  \_ 6:0:0:0 sdc 8:32  [active][undef]

3.  Grow the LUN on the SAN (although, this can happen before steps 1 & 2 
as well)

(NetApp specific:)
lun resize /vol/resize_test/lun 10g

4. # Update partition table

blockdev --rereadpt /dev/sdd
blockdev --rereadpt /dev/sde
blockdev --getsize /dev/sdd
blockdev --getsize /dev/sde

# The values for getsize in both commands must be the SAME
# Record this value
# The above gave us the value 20971520

# Update the dm table for new
vi new

0 2097152 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000 
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000

# Change this to:
0 20971520 multipath 1 queue_if_no_path 0 1 1 round-robin 0 2 1 8:48 1000 
8:64 1000

# Note this is what happened:
# Update the second parameter (num sectors) from 2097152 to 20971520 
(obtained from blockdev --getsize)
# Remove the second pair beginning with the second "round-robin"
# Reduce the number of pairs after "queue_if_no_path" from 2 (0 2 1) to 1 
(0 1 1)

5.  Switch to the new path

multipathd -k
> add path sdd
> add path sde
> del path sdb
> del path sdc

# Note: We are adding the INACTIVE Paths first (sdd+sde) and then removing
# the ACTIVE paths (sdb+sdc). This is VERY important. I tried reversing it 
by
# deleting ACTIVE first then adding INACTIVE. Fortunately, multipathd kept 
one
# of them, refusing to budge.

dmsetup suspend resize_test; dmsetup reload resize_test new; dmsetup 
resume resize_test

# Verify we've switched to the new path:
multipath -l
resize_test (360a98000486e5452576f4336486d4c74) dm-2 NETAPP,LUN
[size=10G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
  \_ 6:0:1:0 sdd 8:48  [active][undef]
  \_ 5:0:1:0 sde 8:64  [active][undef]

# Note: multipath now shows the new size (10G). If it still shows the old 
size, chances are
# that one of the block devices in the pair have not had their partition 
tables updated.
# TURN BACK! AND MAKE SURE TO REREAD THE PARTITION TABLES!


6. Update the block devices on the inactive path

blockdev --rereadpt /dev/sdb
blockdev --rereadpt /dev/sdc
blockdev --getsize /dev/sdb
blockdev --getsize /dev/sdc

7. Re-add the final path

# Update the final dm table

vi final
0 2097152 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000 
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000

# Change to:
0 20971520 multipath 1 queue_if_no_path 0 2 1 round-robin 0 2 1 8:16 4000 
8:32 4000 round-robin 0 2 1 8:48 1000 8:64 1000

# Note this is what happened:
# Update the second parameter (num sectors) from 2097152 to 20971520
# (obtained from blockdev --getsize)

multipathd -k
add path sdb
add path sdc

dmsetup suspend resize_test; dmsetup reload resize_test final; dmsetup 
resume resize_test

multipathd -k
reconfigure

# Note: This reconfigure is *ESSENTIAL* as it forces multipath to remember 
the new size. Without this step, failing one of the paths
# can cause multipath to reuse the previous block device size and cause 
filesystem problems.

8.  Resize the FS from userland

ext2online /dev/mapper/resize_test
(or resize2fs in newer )


Pfew! Done!
I think this should have been shared by us (myself, Gino Ledesma, Michael Hsu) a long time ago.
We probably wanted to make a pretty pretty script to do it for us.

There is a slightly more convoluted recipe if you were bold and foolish 
enough to partition the LUN (like we once were).

Cheers,

-sharif