[rhelv6-list] Megaraid on RHEL6: Replacing the disks in a RAID-5, one after the other..?

[Brian Long (brilong)]

On Sep 16, 2013, at 4:38 PM, vincent at cojot.name wrote:
> 
> Hi everyone,
> 
> I have a Dell T410 with hot-swap bays that hold the following config:
> 
> <snip>
> 
> The 4x3Tb RAID-5 is running out of space and I'd like to replace all of those 3Tb disks with 4Tb disks (Yes, I know it's not economical). I considered rebuilding the array from scratch and restoring the data but the restore window would be too lengthy. I don't have any hot spares, btw, nor empty slots..
> 
> I'm currently wondering if I could just swap the disks one after the other (waiting for the resync to finish in-between swaps) and then extend the Logical Drive.
> 
> Is this doable with such a config? Has someone done this before? What would the procedure look like? I was thinking about something like this (Array #0, LD #1).

I have done this multiple times on LSI-based controllers (9261).  As Michael alluded to, RAID-5 is in it's most vulnerable state when a drive is missing.  It is not uncommon for a second disk to die or drop from the RAID-5 array right when you're trying the process you outlined.  I've not had it happen when I performed this migration on LSI controllers, but I _have_ had this happen when using Linux kernel software RAID and on an older 3Ware hardware RAID controller.  If you dare proceed, make sure you have a current backup since it will be your last resort and then you're stuck with the long restore window.

I like Michael's suggestion of setting up an alternate array or copy of your data.  The other more expensive option would be to acquire a second server with the RAID setup you require, migrate all your apps and data (rsync is your friend).  Then recycle the original server for another task.

/Brian/