[linux-lvm] pvmove hangs
Gergely Imre
gimre at nextra.ro
Wed Apr 27 13:04:20 UTC 2005
let me 'restart' this thread. i ran into another problem. or it's the same?
[root at test etc]# fdisk -l
Disk /dev/sda: 4294 MB, 4294967296 bytes
255 heads, 63 sectors/track, 522 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 1 17 136521 82 Linux swap
/dev/sda2 * 18 267 2008125 83 Linux
/dev/sda3 268 392 1004062+ 8e Linux LVM
/dev/sda4 393 522 1044225 8e Linux LVM
i installed FC2 on sda2, i upgraded the kernel to 2.6.11.7, created a VG
and LV out of sda3+sda4, like this:
[root at test etc]# vgdisplay -v
Finding all volume groups
Finding volume group "test"
--- Volume group ---
VG Name test
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 56
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 1.95 GB
PE Size 4.00 MB
Total PE 499
Alloc PE / Size 244 / 976.00 MB
Free PE / Size 255 / 1020.00 MB
VG UUID E012hQ-KRPN-ygZI-myIs-XfbV-68tt-6S44wH
--- Logical volume ---
LV Name /dev/test/root
VG Name test
LV UUID wqoG5m-X3Fn-Gsaj-8P9t-cSwS-AXi4-IJ5j2P
LV Write Access read/write
LV Status available
# open 1
LV Size 976.00 MB
Current LE 244
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0
--- Physical volumes ---
PV Name /dev/sda3
PV UUID 1FWdvz-Bg30-VGNp-sq2P-HpD3-9x0t-s3QNAV
PV Status allocatable
Total PE / Free PE 245 / 1
PV Name /dev/sda4
PV UUID rwLsxX-3h8f-Z7tc-Jmgv-375C-RFvP-KnxNOj
PV Status allocatable
Total PE / Free PE 254 / 254
so, practically the root LV is on sda3. i moved the system from sda2 to
sda3, now the whole / is on /dev/mapper/test-root, and it's working fine.
[root at test etc]# df -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/mapper/test-root
ext3 983704 638908 294828 69% /
none tmpfs 63464 0 63464 0% /dev/shm
grub.conf:
title Fedora Core (2.6.11.7)
root (hd0,1)
kernel /boot/vmlinuz-2.6.11.7 ro root=/dev/mapper/test-root
initrd /boot/initrd-2.6.11.7.img
so, the /boot directory stayed on /dev/sda2.
fstab:
[root at test etc]# cat /etc/fstab
/dev/mapper/test-root / ext3 defaults 1 1
none /dev/pts devpts gid=5,mode=620 0 0
none /dev/shm tmpfs defaults 0 0
/dev/sda1 swap swap defaults .
.
.
so far so good. now let's say i want to remove sda3. to do this, i need
to pvmove everything from sda3 to sda4. if i run
pvmove -vv /dev/sda3
i get the following:
[root at test root]# pvmove -vv /dev/sda3
Setting global/locking_type to 1
Setting global/locking_dir to /var/lock/lvm
File-based locking enabled.
/dev/sda3: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda1: No label detected
/dev/sda2: No label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
Finding volume group "test"
Locking /var/lock/lvm/V_test WB
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
Archiving volume group "test" metadata.
Creating logical volume pvmove0
Getting target version for mirror
Moving /dev/sda3:0-243 of test/root
Moving 244 extents of logical volume test/root
Finding volume group for uuid
E012hQKRPNygZImyIsXfbV68tt6S44wHwqoG5mX3FnGsaj8P9tcSwSAXi4IJ5j2P
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
Found volume group "test"
Setting activation/missing_stripe_filler to /dev/ioerror
Updating volume group metadata
Creating volume group backup "/etc/lvm/backup/test"
Finding volume group for uuid
E012hQKRPNygZImyIsXfbV68tt6S44wHwqoG5mX3FnGsaj8P9tcSwSAXi4IJ5j2P
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
Found volume group "test"
Locking memory
Suspending test-root
Finding volume group for uuid
E012hQKRPNygZImyIsXfbV68tt6S44wH860Z8k3Rbibp4LTSyFNUkHANkOMh0qdK
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
/dev/sda3: lvm2 label detected
/dev/sda4: lvm2 label detected
Found volume group "test"
Loading test-pvmove0
Setting activation/mirror_region_size to 512
and that's it... i wait around 20 minutes, nothing. no picture, no
sound... i can't do anything, so i have to do a hard reset. there is no
disk activity whatsoever.
after the reset, i quickly log in, and i find (running lvs) that it's
continuing the pvmove, like nothing happened:
lvs[root at test root]# lvs
LV VG Attr LSize Origin Snap% Move Copy%
pvmove0 test p-C-ao 976.00M /dev/sda3 44.67
root test -wI-ao 976.00M
i didn't run anything, still, pvmove continues. after a while it
finishes, and it seems like everything is OK.
[root at test root]# lvs
LV VG Attr LSize Origin Snap% Move Copy%
root test -wi-ao 976.00M
[root at test root]#
[root at test root]# vgdisplay -v
Finding all volume groups
Finding volume group "test"
--- Volume group ---
VG Name test
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 60
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 1.95 GB
PE Size 4.00 MB
Total PE 499
Alloc PE / Size 244 / 976.00 MB
Free PE / Size 255 / 1020.00 MB
VG UUID E012hQ-KRPN-ygZI-myIs-XfbV-68tt-6S44wH
--- Logical volume ---
LV Name /dev/test/root
VG Name test
LV UUID wqoG5m-X3Fn-Gsaj-8P9t-cSwS-AXi4-IJ5j2P
LV Write Access read/write
LV Status available
# open 1
LV Size 976.00 MB
Current LE 244
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:1
--- Physical volumes ---
PV Name /dev/sda3
PV UUID 1FWdvz-Bg30-VGNp-sq2P-HpD3-9x0t-s3QNAV
PV Status allocatable
Total PE / Free PE 245 / 245
PV Name /dev/sda4
PV UUID rwLsxX-3h8f-Z7tc-Jmgv-375C-RFvP-KnxNOj
PV Status allocatable
Total PE / Free PE 254 / 10
now everything is on sda4, i could remove sda3. but here's the thing...
i do a reboot, and i get the following error:
http://terran.noc.astral.ro/pvmove/boot.png
but if i give the root password, /dev/mapper/test-root is mounted, and
if i remount it read-write, it seems to be alright. but if i try to fsck
/dev/mapper/test-root, it still gives me this error
[root at test root]# fsck /dev/mapper/test-root
fsck 1.35 (28-Feb-2004)
e2fsck 1.35 (28-Feb-2004)
fsck.ext3: No such device or address while trying to open
/dev/mapper/test-root
Possibly non-existent or swap device?
[root at test root]#
i look in /dev/mapper:
[root at test root]# cd /dev/mapper
[root at test mapper]# ll
total 0
crw------- 1 root root 10, 63 Apr 27 15:43 control
brw------- 1 root root 253, 0 Apr 27 15:43 test-pvmove0
brw------- 1 root root 253, 1 Apr 27 15:43 test-root
[root at test mapper]#
it did not remove test-pvmove0, after finishing the move. another
strange thing is that now test-root has major/minor 253,1 and
test-pvmove0 has 253,0.
i removed test-*
[root at test mapper]# rm test-pvmove0 test-root
rm: remove block special file `test-pvmove0'? y
rm: remove block special file `test-root'? y
then i did a lvm vgmknodes (i looked this up in /etc/rc.sysinit:)
[root at test mapper]# lvm vgmknodes
[root at test mapper]# ls -la
total 124
drwxr-xr-x 2 root root 4096 Apr 27 15:59 .
drwxr-xr-x 24 root root 118784 Apr 27 15:55 ..
crw------- 1 root root 10, 63 Apr 27 15:43 control
brw------- 1 root root 253, 0 Apr 27 15:59 test-root
it created the test-root node, but with major/minor 253,0, so now the
fsck is working again:
[root at test mapper]# fsck /dev/mapper/test-root
fsck 1.35 (28-Feb-2004)
e2fsck 1.35 (28-Feb-2004)
/dev/mapper/test-root is mounted.
WARNING!!! Running e2fsck on a mounted filesystem may cause
SEVERE filesystem damage.
Do you really want to continue (y/n)? no
check aborted.
if i do a reboot now, everything is alright again, and in fact i got
what i wanted, it moved everything from sda3 to sda4. but what about all
these problems?
it's not over yet :)
now i want to move everything back to sda3. but this time i do a:
[root at test root]# mount / -o remount,noatime
then the moving:
[root at test root]# pvmove -v /dev/sda4
Finding volume group "test"
Archiving volume group "test" metadata.
Creating logical volume pvmove0
Moving 244 extents of logical volume test/root
Found volume group "test"
Updating volume group metadata
Creating volume group backup "/etc/lvm/backup/test"
Found volume group "test"
Found volume group "test"
Loading test-pvmove0
Found volume group "test"
Loading test-root
Checking progress every 15 seconds
/dev/sda4: Moved: 11.9%
/dev/sda4: Moved: 21.7%
/dev/sda4: Moved: 32.4%
/dev/sda4: Moved: 45.5%
/dev/sda4: Moved: 56.1%
/dev/sda4: Moved: 67.6%
/dev/sda4: Moved: 78.3%
/dev/sda4: Moved: 89.3%
/dev/sda4: Moved: 99.6%
/dev/sda4: Moved: 100.0%
Found volume group "test"
Found volume group "test"
Found volume group "test"
Loading test-pvmove0
Found volume group "test"
Loading test-root
Found volume group "test"
Found volume group "test"
Removing temporary pvmove LV
Writing out final volume group after pvmove
Creating volume group backup "/etc/lvm/backup/test"
during to move, i run lvs a couple of times, nothing unusual.
[root at test root]# lvs
LV VG Attr LSize Origin Snap% Move Copy%
pvmove0 test p-C-ao 976.00M /dev/sda4 22.95
root test -wI-ao 976.00M
but what is unusual, is this:
[root at test root]# cd /dev/mapper/
[root at test mapper]# ll
total 0
crw------- 1 root root 10, 63 Apr 27 16:01 control
brw------- 1 root root 253, 1 Apr 27 16:03 test-pvmove0
brw------- 1 root root 253, 0 Apr 27 15:59 test-root
[root at test mapper]#
now it created test-pvmove0 with 253,1, and it didn't touch test-root.
and after pvmove was finished, it removed test-pvmove0 also. now that i
call a proper pvmove ;) i run vgdisplay just to be sure:
[root at test root]# vgdisplay -v
Finding all volume groups
Finding volume group "test"
--- Volume group ---
VG Name test
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 63
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 2
Act PV 2
VG Size 1.95 GB
PE Size 4.00 MB
Total PE 499
Alloc PE / Size 244 / 976.00 MB
Free PE / Size 255 / 1020.00 MB
VG UUID E012hQ-KRPN-ygZI-myIs-XfbV-68tt-6S44wH
--- Logical volume ---
LV Name /dev/test/root
VG Name test
LV UUID wqoG5m-X3Fn-Gsaj-8P9t-cSwS-AXi4-IJ5j2P
LV Write Access read/write
LV Status available
# open 1
LV Size 976.00 MB
Current LE 244
Segments 1
Allocation inherit
Read ahead sectors 0
Block device 253:0
--- Physical volumes ---
PV Name /dev/sda3
PV UUID 1FWdvz-Bg30-VGNp-sq2P-HpD3-9x0t-s3QNAV
PV Status allocatable
Total PE / Free PE 245 / 1
PV Name /dev/sda4
PV UUID rwLsxX-3h8f-Z7tc-Jmgv-375C-RFvP-KnxNOj
PV Status allocatable
Total PE / Free PE 254 / 254
everything is in place (on sda3). sda4 is empty like it should.
could somebody explain this to me? what's happening here?
the box i was playing on is a vmware emulated comp (128MB ram, 4GB hdd),
with a buslogic scsi adapter. i installed fedora core 2 custom, with no
packages selected, after that i did a yum update. kernel 2.6.11.7
vanilla, with no patches at all, compiled with minimum stuff. latest
lvm2 and device-mapper.
output of ps:
[root at test root]# ps ax
PID TTY STAT TIME COMMAND
1 ? S 0:02 init [3]
2 ? SWN 0:00 [ksoftirqd/0]
3 ? SW< 0:00 [events/0]
4 ? SW< 0:00 [khelper]
9 ? SW< 0:00 [kthread]
17 ? SW< 0:00 [kblockd/0]
71 ? SW 0:00 [pdflush]
72 ? SW 0:00 [pdflush]
74 ? SW< 0:00 [aio/0]
73 ? SW 0:00 [kswapd0]
659 ? SW 0:00 [kseriod]
694 ? SW 0:00 [scsi_eh_0]
715 ? SW< 0:00 [kcryptd/0]
716 ? SW< 0:00 [kmirrord/0]
751 ? SW 0:00 [kjournald]
1722 ? S 0:00 syslogd -m 0
1726 ? S 0:00 klogd -x
1749 ? S 0:00 /usr/sbin/sshd
1761 ? S 0:00 crond
1785 tty1 S 0:00 /sbin/mingetty tty1
1807 tty2 S 0:00 /sbin/mingetty tty2
1818 tty3 S 0:00 /sbin/mingetty tty3
1819 tty4 S 0:00 /sbin/mingetty tty4
1906 tty5 S 0:00 /sbin/mingetty tty5
1937 tty6 S 0:00 /sbin/mingetty tty6
1972 ? R 0:00 sshd: root at pts/0
1974 pts/0 S 0:00 -bash
2010 ? S 0:00 sshd: root at pts/1
2012 pts/1 S 0:00 -bash
2047 pts/0 R 0:00 ps ax
(sorry for the long mail;)
More information about the linux-lvm
mailing list