grub failure to update MBR (using RAID and LVM)

[Shawn Iverson]

Greetings again,

I have yet another issue with my server.  I cannot update my GRUB menu.
I have a RAID 1 set up as follows on FC3 with latest official updates:

/dev/hda, /dev/hdb, and /dev/hdc, with hda and hdb containing two
mirrors and /dev/hdc as a hot spare.
/dev/md0 is /boot with 1GB of space on and ext3 filesystem
/dev/md1 is a linux LVM with a separate logical volume for each major
mount point (/usr, /var, /, /home, etc.) using ext3 on a single volume
group.

Grub is currently installed across all three MBRs on each drive, and the
RAIDs are healthy. 

I attempt to install GRUB on /dev/hda (hd0) using the grub interactive
mode:

grub> setup (hd0)
 Checking if "/boot/grub/stage1" exists... no
 Checking if "/grub/stage1" exists... yes
 Checking if "/grub/stage2" exists... yes
 Checking if "/grub/e2fs_stage1_5" exists... yes
 Running "embed /grub/e2fs_stage1_5 (hd0)"...  16 sectors are embedded.
succeeded
 Running "install /grub/stage1 (hd0) (hd0)1+16 p (hd0,0)/grub/stage2
/grub/grub.conf"... succee
ded
Done.

I get the same results with the other two drives, hd1 and hd2.

I am currently using grub-0.95-3.

Here is my /boot/grub/grub.conf and /boot/grub/device.map:

[admin at mrtg grub]# cat device.map
(fd0)   /dev/fd0
(hd0)   /dev/hda
(hd1)   /dev/hdb
(hd2)   /dev/hdc

# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this
file
# NOTICE:  You have a /boot partition.  This means that
#          all kernel and initrd paths are relative to /boot/, eg.
#          root (hd0,0)
#          kernel /vmlinuz-version ro root=/dev/Volume00/root
#          initrd /initrd-version.img
boot=/dev/hda /dev/hdb /dev/hdc
default=0
timeout=10
splashimage=(hd0,0)/grub/splash.xpm.gz
title Fedora Core (2.6.10-1.766_FC3)
        root (hd0,0)
        kernel /vmlinuz-2.6.10-1.766_FC3 ro root=/dev/Volume00/root
        initrd /initrd-2.6.10-1.766_FC3.img
title Fedora Core (2.6.9-1.681_FC3)
        root (hd0,0)
        kernel /vmlinuz-2.6.9-1.681_FC3 ro root=/dev/Volume00/root
acpi=off
        initrd /initrd-2.6.9-1.681_FC3.img
title Fedora Core (2.6.5-1.358)
        root (hd0,0)
        kernel /vmlinuz-2.6.9-1.667 ro root=/dev/Volume00/root acpi=off
        initrd /initrd-2.6.9-1.667.img

My BIOS settings are set to boot in this order:  IDE0, IDE1, IDE2.  I
isolated the boot to IDE0 for the sake of troubleshooting this, though.
Boot other devices is disabled.

Instead of getting the menu with the three choices, I still see my older
menu containing only the two older kernels after rebooting.  I never had
this much trouble with a boot loader before, and I am even considering
switching it over to LILO because it seems to handle software RAID so
much better, that is, if I cannot resolve this issue.

--

Shawn Iverson
Technology Associate
MCP W2K3S and W2KP, Linux+, Network+, A+
New Castle Community School Corporation
shawn at nccsc.k12.in.us