[linux-lvm] Issues with > 2TB PV with old GPT; how to build a partition around it

[Rich Graves]

Platform: RHEL4 2.6.9-55.0.9.ELsmp x86_64.

In part, this is a redirect (not cross-post) from https://www.redhat.com/archives/nahant-list/2007-October/msg00020.html

I have a single-PV VG on a 2.7TB LUN on an Apple Xserve RAID, multipath fibre channel-attached.

The 2.7TB LUN regularly "disappears," I think whenever multipath does anything (including regular multipathd checks). Currently a filesystem on that LUN is mounted RW and shows up in dmsetup, but not in lvscan. The relevant /etc/lvm/backup/ file has this fascinating tidbit:

                        dev_size = 1565605888   # 746.539 Gigabytes
                        pe_start = 384
                        pe_count = 715401       # 2.72904 Terabytes

The physical extents are larger than the device? Looks like it wrapped at 2TB. Why did it let me do that? I knew that I couldn't create a > 2TB partition with fdisk, but I didn't understand that on a 64-bit OS, I couldn't point LVM2 at a > 2TB raw disk. (Is that in fact the problem?)

More ugly history: The LUN had been used by a Windows machine, with a GUID partition table. I overwrote the first several blocks with zeroes and created a single-PV VG taking up the whole 2.7TB, then pointed pvcreate at the parent devive-mapper path for the disk (not the partitions). Unfortunately, I did not blow away the backup GPT, so those partitions popped up again upon reboot. Oops. But that history is probably beside the point.

I would like to resolve this without recreating the filesystem, which has a large amount of data on it. The filesystem works just fine (doesn't it?), as long as I don't reboot. :-|

The start of the filesystem is easy to find, and it is exactly 2TB long (I created the LV with padding), so there should be no problem:

[root at mail2 ~]# dd if=/dev/sdd bs=512 count=1 skip=384 count=3 of=/tmp/fs
3+0 records in
3+0 records out
[root at mail2 ~]# file /tmp/fs
/tmp/fs: Linux rev 1.0 ext3 filesystem data (needs journal recovery) (large file
s)

So, [how] can I create a partition with parted that happens to land at the right blocks? Can I safely reduce the "pe_start" parameter of the LVM by the partition overhead, which appears to be 34 blocks, then pvcreate --restorefile the modified config?

# Generated by LVM2: Thu Jul 19 23:23:35 2007

contents = "Text Format Volume Group"
version = 1

description = "Created *after* executing 'lvcreate -L2T -n lvZimbraBackup1 vgZimOffsite'"

creation_host = "mail2.its.carleton.edu"        # Linux mail2.its.carleton.edu 2.6.9-55.0.2.ELsmp #1 SMP Tue Jun 12 17:58:20 EDT 2007 x86_64 creation_time = 1184905415      # Thu Jul 19 23:23:35 2007

vgZimOffsite {
        id = "vqX3RX-7JVH-UVCE-4Zfh-xa6F-hak8-gih0v8"
        seqno = 2
        status = ["RESIZEABLE", "READ", "WRITE"]
        extent_size = 8192              # 4 Megabytes
        max_lv = 0
        max_pv = 0

        physical_volumes {

                pv0 {
                        id = "9yEHDP-hVi2-W72J-wWAP-loRk-6Do5-5SQwe0"
                        device = "/dev/dm-4"    # Hint only

                        status = ["ALLOCATABLE"]
                        dev_size = 1565605888   # 746.539 Gigabytes
                        pe_start = 384
                        pe_count = 715401       # 2.72904 Terabytes
                }
        }

        logical_volumes {

                lvZimbraBackup1 {
                        id = "GCeeRm-oKFZ-Rqz9-vM0w-zegj-PljH-bAv9qm"
                        status = ["READ", "WRITE", "VISIBLE"]
                        segment_count = 1

                        segment1 {
                                start_extent = 0
                                extent_count = 524288   # 2 Terabytes

                                type = "striped"
                                stripe_count = 1        # linear

                                stripes = [
                                        "pv0", 0
                                ]
                        }
                }
        }
}
-- 
Rich Graves http://claimid.com/rcgraves
Carleton.edu Sr UNIX and Security Admin
CMC135: 507-646-7079 Cell: 952-292-6529