[linux-lvm] Fail to establish LVM2 RAID-1 to an iSCSI Disk after Rebooting RHEL-6

[Shi Jin]

Hi there,

I have a slightly non-traditional setup of a RHEL-6.4 box where it is
mirrored using LVM2 RAID-1 to a iSCSI target which is connected by the
RHEL-6 itself via open-iscsi after it boots.

I am not doing any ramdisk change to the box to connect to the iSCSI
while kernel is loading since I believe there is a way to do
incremental resynch of the RAID-1 after Linux boots.

Before the reboot, I have a healthy raid-1 between local disk (sda)
and iscsi disk (sdb):

[root at dg-test-rhel2 LinuxOnboarding]#  lvs -a -o +seg_pe_ranges
  LV                 VG             Attr      LSize  Pool Origin Data%
 Move Log Cpy%Sync Convert PE Ranges
  lv_root            vg_dgtestrhel1 rwi-aor-- 15.57g
            100.00         lv_root_rimage_0:0-3985
lv_root_rimage_1:0-3985
  [lv_root_rimage_0] vg_dgtestrhel1 iwi-aor-- 15.57g
                           /dev/sda2:0-3985
  [lv_root_rimage_1] vg_dgtestrhel1 iwi-aor-- 15.57g
                           /dev/sdb2:1-3986
  [lv_root_rmeta_0]  vg_dgtestrhel1 ewi-aor--  4.00m
                           /dev/sda2:4866-4866
  [lv_root_rmeta_1]  vg_dgtestrhel1 ewi-aor--  4.00m
                           /dev/sdb2:0-0
  lv_swap            vg_dgtestrhel1 rwi-aor--  3.44g
            100.00         lv_swap_rimage_0:0-879
lv_swap_rimage_1:0-879
  [lv_swap_rimage_0] vg_dgtestrhel1 iwi-aor--  3.44g
                           /dev/sda2:3986-4865
  [lv_swap_rimage_1] vg_dgtestrhel1 iwi-aor--  3.44g
                           /dev/sdb2:3988-4867
  [lv_swap_rmeta_0]  vg_dgtestrhel1 ewi-aor--  4.00m
                           /dev/sda2:4867-4867
  [lv_swap_rmeta_1]  vg_dgtestrhel1 ewi-aor--  4.00m
                           /dev/sdb2:3987-3987

[root at dg-test-rhel2 LinuxOnboarding]# pvs -v
    Scanning for physical volume names
  PV         VG             Fmt  Attr PSize  PFree   DevSize PV UUID
  /dev/sda2  vg_dgtestrhel1 lvm2 a--  19.51g 504.00m  19.51g
MNbVOF-3d8V-kOFB-Y1X4-6sgw-Ny24-9Knicq
  /dev/sdb2  vg_dgtestrhel1 lvm2 a--  21.87g   2.86g  21.88g
9hlSsu-I8Rc-2IMl-tQyb-NAf0-RqWO-LJBkRf
[root at dg-test-rhel2 LinuxOnboarding]# vgs
  VG             #PV #LV #SN Attr   VSize  VFree
  vg_dgtestrhel1   2   2   0 wz--n- 41.38g 3.35g

After I reboot the box, Linux will automatically connects to the iscsi
LUN but the mirror is broken (if I do any write, I only see disk
activity on sda):
[root at dg-test-rhel2 ~]#  lvs -a -o +seg_pe_ranges
  LV                 VG             Attr      LSize  Pool Origin Data%
 Move Log Cpy%Sync Convert PE Ranges
  lv_root            vg_dgtestrhel1 rwi-aor-- 15.57g
            100.00         lv_root_rimage_0:0-3985
lv_root_rimage_1:0-3985
  [lv_root_rimage_0] vg_dgtestrhel1 iwi-aor-- 15.57g
                           /dev/sda2:0-3985
  [lv_root_rimage_1] vg_dgtestrhel1 iwi-a-r-- 15.57g
                           /dev/sdb2:1-3986
  [lv_root_rmeta_0]  vg_dgtestrhel1 ewi-aor--  4.00m
                           /dev/sda2:4866-4866
  [lv_root_rmeta_1]  vg_dgtestrhel1 ewi-a-r--  4.00m
                           /dev/sdb2:0-0
  lv_swap            vg_dgtestrhel1 rwi-aor--  3.44g
            100.00         lv_swap_rimage_0:0-879
lv_swap_rimage_1:0-879
  [lv_swap_rimage_0] vg_dgtestrhel1 iwi-aor--  3.44g
                           /dev/sda2:3986-4865
  [lv_swap_rimage_1] vg_dgtestrhel1 iwi-a-r--  3.44g
                           /dev/sdb2:3988-4867
  [lv_swap_rmeta_0]  vg_dgtestrhel1 ewi-aor--  4.00m
                           /dev/sda2:4867-4867
  [lv_swap_rmeta_1]  vg_dgtestrhel1 ewi-a-r--  4.00m
                           /dev/sdb2:3987-3987
[root at dg-test-rhel2 ~]# pvs
  PV         VG             Fmt  Attr PSize  PFree
  /dev/sda2  vg_dgtestrhel1 lvm2 a--  19.51g 504.00m
  /dev/sdb2  vg_dgtestrhel1 lvm2 a--  21.87g   2.86g
[root at dg-test-rhel2 ~]# vgs
  VG             #PV #LV #SN Attr   VSize  VFree
  vg_dgtestrhel1   2   2   0 wz--n- 41.38g 3.35g


Please note that the  [lv_root_rimage_1] and  [lv_root_rmeta_1]
devices are no longer open even though LVM reports there is a mirror.

I am looking for a simple LVM command that could force the RAID-1
mirror to resynch (ideally incrementally since we have the bitmap
device which should track all the un-synched data). I am able to
convert the LV to Linear volumes and re-create the mirror once again
but that is very lame.

Please help!
Many thanks,

Shi
PS. My OS information

[root at dg-test-rhel2 ~]# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.4 (Santiago)
[root at dg-test-rhel2 ~]# uname -a
Linux dg-test-rhel2.doe.local 2.6.32-358.el6.x86_64 #1 SMP Tue Jan 29
11:47:41 EST 2013 x86_64 x86_64 x86_64 GNU/Linux
[root at dg-test-rhel2 ~]# lvm version
  LVM version:     2.02.98(2)-RHEL6 (2012-10-15)
  Library version: 1.02.77-RHEL6 (2012-10-15)
  Driver version:  4.23.6
[root at dg-test-rhel2 ~]#