Replacing disk in Linux Software RAID 1

[Nigel Wade]

Michael Mansour wrote:
> Hi Nigel,
> 
>  --- Nigel Wade <nmw at ion.le.ac.uk> wrote: 
> 
>>Michael E. Webster wrote:
>>
>>>Michael,
>>>
>>>Probably - I went with "ghost" because I had a
>>
>>boot floppy handy.
>>
>>You should not need to ghost. Add the new drive as a
>>hot spare and md should 
>>  take care of building the new drive whilst the
>>system is running (it does 
>>need to be partitioned correctly first).
> 
> 
> I wish I knew this before I did the work :) ... I
> ended up using ghost but it would have been easier if
> not I think.
> 
> What I ended up doing was purchasing two new drives,
> popping one in, ghosting, then the next (while
> removing the old) and re-ghosting. At the same time I
> decided to use Partition Magic to extend my "/home"
> partition as I needed the space (and since I had both
> original drives I didn't mind stuffing things up since
> I could then just pop in the old drives).
> 
> What I then found was that I needed to boot off rescue
> and re-sync the mirrors.
> 
> When this completed, I re-booted and /home refused to
> mount complaining that the partition table size and
> the physical size were different. After alot of
> googling, I found out this occured because software
> raid (which partition magic doesn't recognise - I can
> explain more here if requested) reserves some space at
> the end of the partition for itself. I had to use the
> "resize2fs" command to resize the md before it would
> successfully boot.
> 
> 
>>>The main thing is to get the MBR from that first
>>
>>drive - without it,
>>
>>>you're out of luck.  
> 
> 
> Well, what I did was create a grub 0.93 boot disk,
> from there I already had my grub.conf documented and
> it was a simple procedure to:
> 
> grub> root (hd0,1)
> grub> kernel blah blah
> grub> initrd blah blah
> grub> setup (hd0)
> 
> to get grub back on there.
> 
> 
>>>In software RAID, the MBR is only written to the
>>
>>first drive, which
>>
>>>really doesn't make sense.  I would think that if
>>
>>you had the first
>>
>>>drive fail, you should be able to boot off the
>>
>>second one - IMHO.
>>
>>>
>>>Mike.
>>>
>>>
>>
>>I don't think that's right.
> 
> 
> That is right actually. What you'll find with grub is
> if you pull out the first disk, the system won't boot
> from the second. You have to put grub back onto the
> MBR of the second to boot from it.
> 

Ok, I'll state it more clearly. It's not totally correct, it's only true for 
grub. If you use lilo it writes MBRs to both drives such that either can be 
used to boot without the other present.

If you use grub, you need to write MBRs to both drives. If the bootable 
drive fails you need to physically relocate the other drive so that it 
becomes the bootable drive. You can install, for example, grub in both 
/dev/hda and /dev/hdc, but only /dev/hda would actually be bootable. The 
reason is that the grub configuration files are stored in the mirrored 
filesystem, and these tell it where to get the kernel etc from. There cannot 
be different versions for each disk if the filesytem is mirrored. With it's 
current configuration, grub cannot be made properly raid-aware.

Lilo, OTOH, only requires the physical sector of the disk in the MBR to 
which it should jump. Lilo is raid-aware and writes MBRs to each disk in the 
raid if you give boot= as a raid partition, and the boot sequence for each 
disk will jump to the correct location on that disk.

If you want to have mirrored /boot and /, you are better off using lilo.

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw at ion.le.ac.uk
Phone :     +44 (0)116 2523548, Fax : +44 (0)116 2523555