Help with RAID 5 Failure

[Clive Messer]

Can someone wiser than me help get my RAID5 array back online?

I have a raid 5 array /dev/md0, with 4 300GB disks, 
comprising /dev/sda1, /dev/sdb1, /dev/sdc1 & /dev/sdd1. One of the disks 
(/dev/sda1) was failed from the array last night. I shut down this morning 
expecting to be able to replace the disc, reboot, fdisk /dev/sda1, and 
mdadm /dev/md0 --add /dev/sda1. However, the array will not come back online. 
It should with 3 original disks out of the 4, right? The array is assembled, 
but inactive. If I try to start it....

# mdadm --run /dev/md0
mdadm: failed to run array /dev/md0: Input/output error

Here is some info.......

# cat /proc/mdstat
Personalities : [raid5]
md0 : inactive sdb1[1] sdd1[3] sdc1[2]
      879148800 blocks

unused devices: <none>

# mdadm --detail /dev/md0
/dev/md0:
        Version : 00.90.03
  Creation Time : Sat Feb 18 01:51:12 2006
     Raid Level : raid5
    Device Size : 293049600 (279.47 GiB 300.08 GB)
   Raid Devices : 4
  Total Devices : 3
Preferred Minor : 0
    Persistence : Superblock is persistent

    Update Time : Sun Mar 19 11:54:53 2006
          State : active, degraded
 Active Devices : 3
Working Devices : 3
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 64K

           UUID : d72f7bc1:dbf26498:7b3f419f:28ae8a1c
         Events : 0.301864

    Number   Major   Minor   RaidDevice State
       0       0        0        -      removed
       1       8       17        1      active sync   /dev/sdb1
       2       8       33        2      active sync   /dev/sdc1
       3       8       49        3      active sync   /dev/sdd1

# mdadm --examine /dev/sdb1
/dev/sdb1:
          Magic : a92b4efc
        Version : 00.90.03
           UUID : d72f7bc1:dbf26498:7b3f419f:28ae8a1c
  Creation Time : Sat Feb 18 01:51:12 2006
     Raid Level : raid5
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0

    Update Time : Sun Mar 19 11:54:53 2006
          State : active
 Active Devices : 3
Working Devices : 3
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 99cbf09f - correct
         Events : 0.301864

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     1       8       17        1      active sync   /dev/sdb1

   0     0       0        0        0      removed
   1     1       8       17        1      active sync   /dev/sdb1
   2     2       8       33        2      active sync   /dev/sdc1
   3     3       8       49        3      active sync   /dev/sdd1


# mdadm --examine /dev/sdc1
/dev/sdc1:
          Magic : a92b4efc
        Version : 00.90.03
           UUID : d72f7bc1:dbf26498:7b3f419f:28ae8a1c
  Creation Time : Sat Feb 18 01:51:12 2006
     Raid Level : raid5
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0

    Update Time : Sun Mar 19 11:54:53 2006
          State : active
 Active Devices : 3
Working Devices : 3
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 99cbf0b1 - correct
         Events : 0.301864

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     2       8       33        2      active sync   /dev/sdc1

   0     0       0        0        0      removed
   1     1       8       17        1      active sync   /dev/sdb1
   2     2       8       33        2      active sync   /dev/sdc1
   3     3       8       49        3      active sync   /dev/sdd1

# mdadm --examine /dev/sdd1
/dev/sdd1:
          Magic : a92b4efc
        Version : 00.90.03
           UUID : d72f7bc1:dbf26498:7b3f419f:28ae8a1c
  Creation Time : Sat Feb 18 01:51:12 2006
     Raid Level : raid5
   Raid Devices : 4
  Total Devices : 4
Preferred Minor : 0

    Update Time : Sun Mar 19 11:54:53 2006
          State : active
 Active Devices : 3
Working Devices : 3
 Failed Devices : 1
  Spare Devices : 0
       Checksum : 99cbf0c3 - correct
         Events : 0.301864

         Layout : left-symmetric
     Chunk Size : 64K

      Number   Major   Minor   RaidDevice State
this     3       8       49        3      active sync   /dev/sdd1

   0     0       0        0        0      removed
   1     1       8       17        1      active sync   /dev/sdb1
   2     2       8       33        2      active sync   /dev/sdc1
   3     3       8       49        3      active sync   /dev/sdd1


Regards

Clive
-- 
Clive Messer <clive at vacuumtube.org.uk>