[linux-lvm] f*cked up metadata on 1of3 LVM-disks

[BigMac]

Hi lists!

My fileserver stored it's data on 3 physical disks,building a
volumegroup with lvm2.
The mashine was running ubuntu dapper drake for a long time without any
problems but I decided to switch to ubuntu feisty fawn. Switching from
debian to dapper drake I installed and removed quite a lot of packages,
leaving a messy system, so I thought a new and clean installation with
feisty fawn might be a great idea.

box config with dapper drake:

hda: 120GB (PATA) systemdisk
hdb: unused
hdc: 250GB (PATA) part of the LVM-array
hdd: 250GB (PATA) part of the LVM-array
sda: 400GB (SATA) part of the LVM-array
sdb: DVD-RW (PATA on additional PCI-controller UDMA-100)

The additional IDE-controller was installed to be able to install even
more disks, but as I won't buy any new PATA stuff the controller was
dispensable.

new box config:

hda: 120GB (PATA) systemdisk
hdb: DVD-RW
hdc: 250GB (PATA) part of the LVM-array
hdd: 250GB (PATA) part of the LVM-array
sda: 400GB (SATA) part of the LVM-array

I exported the vg before shutting down dapper drake and while installing
festy fawn to the box I - no idea what did me do so - installed grub to
hdc. Ooops, this disk was part of the LVM array and so grub f*cked up
the lvm meta-data on hdc.

After installing grub to hda the box is running fine, despite the LVM
which can't find one of the vgs volumes:

Code:

bigmac at knecht:~$ sudo pvscan
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
PV /dev/hdd         VG vgdata   lvm2 [232.88 GB / 0    free]
PV unknown device   VG vgdata   lvm2 [232.88 GB / 0    free]
PV /dev/sda         VG vgdata   lvm2 [372.61 GB / 4.00 MB free]
Total: 3 [838.38 GB] / in use: 3 [838.38 GB] / in no VG: 0 [0   ]

bigmac at knecht:~$ sudo pvdisplay
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
--- Physical volume ---
PV Name               /dev/hdd
VG Name               vgdata
PV Size               232.88 GB / not usable 0
Allocatable           yes (but full)
PE Size (KByte)       4096
Total PE              59618
Free PE               0
Allocated PE          59618
PV UUID               locW5S-HNFK-b1WW-iOLt-olHl-2SNd-fmXKC8

--- Physical volume ---
PV Name               unknown device
VG Name               vgdata
PV Size               232.88 GB / not usable 0
Allocatable           yes (but full)
PE Size (KByte)       4096
Total PE              59618
Free PE               0
Allocated PE          59618
PV UUID               9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81

--- Physical volume ---
PV Name               /dev/sda
VG Name               vgdata
PV Size               372.61 GB / not usable 0
Allocatable           yes
PE Size (KByte)       4096
Total PE              95388
Free PE               1
Allocated PE          95387
PV UUID               YUJcsF-3XP7-OrbO-pITp-5z96-gw1A-DI11QS


bigmac at knecht:~$ sudo vgscan
Reading all physical volumes.  This may take a while...
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find all physical volumes for volume group vgdata.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find all physical volumes for volume group vgdata.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find all physical volumes for volume group vgdata.
Couldn't find device with uuid '9kpWwu-LdwT-YiqV-DRPg-kSAN-plYN-v1Eq81'.
Couldn't find all physical volumes for volume group vgdata.
Volume group "vgdata" not found

I tried multiple Live-CDs like gparted (with testdisk), Ultimate Boot-CD
and SystemRescueCd, but none could help.

Any idea what to do to get the lvm up and running?
How to restore the metadata on hdc?

Best Regards,

BigMac