[linux-lvm] question about corrupted lvm2 volumes
Jeremy Johnson
jazz_johnson at verizon.net
Fri Jun 9 19:45:51 UTC 2006
I have an amd64 computer running 64-bit Gentoo.
I had a pair of 160G hard drives in a raid1 mirror configuration.
I recently added another 160G harddrive which I set up as
a lvm2 physical volume: `lvcreate /dev/sda && vgcreate extra /dev/sda`
I then created a logical volume on /dev/extra and copied my old filesystem
to this new volume:
#lvcreate -L50G -nbackup extra
#mkreiserfs /dev/extra/backup
#mount /dev/extra/backup /mnt/backup
#BACKUP / /mnt/backup
I then repartitioned my original pair of harddrives
#cfdisk; sfdisk -d /dev/sdb | sfdisk /dev/sdc
and repartitioned:
I set up 4 partitions
/dev/md1 /dev/sd{b,c}1 /boot ext2
/dev/sd{b,c}2 swap sw
/dev/md3 /dev/sd{b,c}3 / reiserfs
/dev/md4 /dev/sd{b,c}4 LVM2 main
/dev/main/{archives ,home ,opt ,portage ,tmp ,usr ,var vartmp}
I initialized filesystems on /dev/main/{LVM2 volumes}
and mounted /dev/md3 on /mnt/newroot, created mount points and
mounted LVM2 volumes on /mnt/newroot/{usr,opt,var,...etc}
and restored from my /mnt/backup to my /mnt/newroot
I then chroot'd into my new root and edited /etc/{fstab,mdadm.conf,lvm/}
and I setup grub into my MBR on /dev/sd{b,c} <=> (hd1) and (hd2):
grub> root (hd1,0)
grub> setup (hd1)
grub> root (hd2,0)
grub> setup (hd2)
My problem is that after booting into my new system
3 of my LVM2 partitions {usr,opt,archives} get corrupted during booting.
If I then reinitizialize their filesystems and restore from backup
my system seems to be working OK. If I then reboot to a livecd
I can then mount all my LVM2 partitions without error.
But if I reboot to my system these same 3 LVM2 volumes get corrupted
and when I try to mount the partitions I get error message
"Operation not permitted"
So running e.g.
#reiserfck /dev/main/opt:
Replaying journal ...
Reiserfs journal /dev/main/opt in blocks [18..8211]
0 transactions replayed
Checking internal tree on 1 1(of14) 1(of 170) block 8211
The level of the node (25938) is not correct (1) expected.
the problem in the internal node occurred (8211)
whole subtree skipped
finished
Comparing bitmaps ..vpf--10640: The on-disk and the correct bitmaps differs
Bad nodes were found, semantic pass skipped
1 found corruption can be fixed when running with --rebuild-tree
I've tried lvremove'ing these volumes and re-lvcreate'ing them but haven't
yet fixed the problem. I've built my 2.6.16-gentoo-r3 kernel with
device-mapper + lvm2 + raid0 + raid1 built-in:
#zless /proc/config.gz
#
# Multi-device support (RAID and LVM)
#
CONFIG_MD=y
CONFIG_BLK_DEV_MD=y
CONFIG_MD_LINEAR=m
CONFIG_MD_RAID0=y
CONFIG_MD_RAID1=y
CONFIG_MD_RAID10=m
CONFIG_MD_RAID5=m
CONFIG_MD_RAID6=m
# CONFIG_MD_MULTIPATH is not set
# CONFIG_MD_FAULTY is not set
CONFIG_BLK_DEV_DM=y
CONFIG_DM_CRYPT=m
CONFIG_DM_SNAPSHOT=m
CONFIG_DM_MIRROR=m
CONFIG_DM_ZERO=m
# CONFIG_DM_MULTIPATH is not set
# CONFIG_BLK_DEV_DM_BBR is not set
I've removed my old /etc/raidtab
and I've done a `mdadm --details --scan >>/etc/mdadm.conf`
and I've modified my filter in /etc/lvm/lvm.conf: filter =
[ "a|/dev/sda|", "a|/dev/md4|", "a|/dev/main/*|", "a|/dev/extra/*|", "r/.*/" ]
This is tail of what `vgdisplay` says:
Finding volume group "main"
Opened /dev/md4 RO O_DIRECT
/dev/md4: block size is 4096 bytes
/dev/md4: lvm2 label detected
/dev/md4: Found metadata at 90624 size 2745 for main
(CcE5mw-EKsk-u2QZ-tqZp-26YU-QTx6-4YIlys)
/dev/md4: lvm2 label detected
/dev/md4: Found metadata at 90624 size 2745 for main
(CcE5mw-EKsk-u2QZ-tqZp-26YU-QTx6-4YIlys)
Read main metadata (34) from /dev/md4 at 90624 size 2745
/dev/md4 0: 0 7680: archives(11887:0)
/dev/md4 1: 7680 512: portage(0:0)
/dev/md4 2: 8192 2560: home(0:0)
/dev/md4 3: 10752 913: archives(19567:0)
/dev/md4 4: 11665 1024: opt(0:0)
/dev/md4 5: 12689 111: NULL(0:0)
/dev/md4 6: 12800 1536: vartmp(0:0)
/dev/md4 7: 14336 512: tmp(0:0)
/dev/md4 8: 14848 1024: NULL(0:0)
/dev/md4 9: 15872 1024: var(0:0)
/dev/md4 10: 16896 7680: usr(0:0)
/dev/md4 11: 24576 11887: archives(0:0)
--- Volume group ---
VG Name main
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 34
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 8
Getting device info for main-portage
dm version O [16384]
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlysxJOAb8ERUBxCzQ7bgLrdPUC8d9eQCzdk O
[16384]
Getting device info for main-home
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlys0lnabC1A2xWwwWejywcctENDtBxn4exj O
[16384]
Getting device info for main-vartmp
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlysXXkM5IYLIvUyx0h0I8kNLK1R8i2fyPmT O
[16384]
Getting device info for main-tmp
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlysTpyh1z5fxA98aPlbuLGKYCVuER3kvDtB O
[16384]
Getting device info for main-var
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlyse9ahmisyqPN2H6vKujCaXan2X1u7N9WV O
[16384]
Getting device info for main-usr
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlysRiNQCmmlDqipc0gvIF3R5huzf3Q9rdbg O
[16384]
Getting device info for main-archives
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlysRWgvL4RZO6izvsBHqGez3V5h5KUsBsLB O
[16384]
Getting device info for main-opt
dm info
LVM-CcE5mwEKsku2QZtqZp26YUQTx64YIlysHBTkmIQSC4Rb6xZ7J13gISkRe4InoFBf O
[16384]
Open LV 7
Max PV 0
Cur PV 1
Act PV 1
VG Size 142.43 GB
PE Size 4.00 MB
Total PE 36463
Alloc PE / Size 35328 / 138.00 GB
Free PE / Size 1135 / 4.43 GB
VG UUID CcE5mw-EKsk-u2QZ-tqZp-26YU-QTx6-4YIlys
Read volume group main from /etc/lvm/backup/main
Unlocking /var/lock/lvm/V_main
Closed /dev/md4
Locking /var/lock/lvm/V_extra RB
Finding volume group "extra"
Opened /dev/sda RO O_DIRECT
/dev/sda: block size is 4096 bytes
/dev/sda: lvm2 label detected
/dev/sda: Found metadata at 24064 size 1810 for extra
(KboSee-hy8G-4NV9-jHfb-PiXt-m9wI-MewC6X)
/dev/sda: lvm2 label detected
/dev/sda: Found metadata at 24064 size 1810 for extra
(KboSee-hy8G-4NV9-jHfb-PiXt-m9wI-MewC6X)
Read extra metadata (12) from /dev/sda at 24064 size 1810
/dev/sda 0: 0 2560: distfiles(0:0)
/dev/sda 1: 2560 2560: archives(0:0)
/dev/sda 2: 5120 5120: archives-bkup(0:0)
/dev/sda 3: 10240 10240: NULL(0:0)
/dev/sda 4: 20480 10240: backup(0:0)
/dev/sda 5: 30720 3840: oldroot(0:0)
/dev/sda 6: 34560 3596: NULL(0:0)
--- Volume group ---
VG Name extra
System ID
Format lvm2
Metadata Areas 1
Metadata Sequence No 12
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 5
Getting device info for extra-distfiles
dm info
LVM-KboSeehy8G4NV9jHfbPiXtm9wIMewC6X7lJNShC8kMJ5uhzo9jBpGT7IBUp9HGYC O
[16384]
Getting device info for extra-archives
dm info
LVM-KboSeehy8G4NV9jHfbPiXtm9wIMewC6XdArNebRyzy0ix2AxgqO7MH0di9u3FcIy O
[16384]
Getting device info for extra-backup
dm info
LVM-KboSeehy8G4NV9jHfbPiXtm9wIMewC6XHD0VXSAb2YOiMjtomSa2pwCXnNcfh6rN O
[16384]
Getting device info for extra-oldroot
dm info
LVM-KboSeehy8G4NV9jHfbPiXtm9wIMewC6X6ZQkzscIHkbgP4KGCGhG1reDBm3dcuUt O
[16384]
Getting device info for extra-archives--bkup
dm info
LVM-KboSeehy8G4NV9jHfbPiXtm9wIMewC6XOM3JPV7QVyvjg35osPHriSixzA89rU4k O
[16384]
Open LV 2
Max PV 0
Cur PV 1
Act PV 1
VG Size 149.05 GB
PE Size 4.00 MB
Total PE 38156
Alloc PE / Size 24320 / 95.00 GB
Free PE / Size 13836 / 54.05 GB
VG UUID KboSee-hy8G-4NV9-jHfb-PiXt-m9wI-MewC6X
Read volume group extra from /etc/lvm/backup/extra
Unlocking /var/lock/lvm/V_extra
Closed /dev/sda
Dumping persistent device cache to /etc/lvm/.cache
Wiping internal VG cache
====================================
Does anybody have a clue as to what might be causing
a few (but not all) of the volumes on my /dev/main <==> /dev/md4
to get corrupted during booting?
Does anything look strange in my vgdisplay? or lvm.conf filter?
Nothing should even be written to /usr,/opt,or /archives during bootup
so what could corrupt these LVM2 volumes?
THANKS for any insights.
More information about the linux-lvm
mailing list