Remote Rescue - PXE, SSH, and a remote console?

[Ryan Wilson]

Hi,

I've got a Linux server at my parent's house. It had some hardware troubles
and I shipped them a new computer and asked them to move the hard drive
over. That worked out great, except now Fedora 8 won't boot. I think I know
why (I think I need to rebuild my initrd), but how can I do that from
remote? Unfortunately, due to time zone issues, I can't talk with my parents
directly, just via e-mail. This has led to some really slow reboot cycles...
They put in a live cd, configure the box for remote login and e-mail me.
Later (maybe the next day), I login make a change and reboot. The system
doesn't come back up so I have to e-mail them to tell me the error and have
them put in the live cd again. This has repeated several times. One reboot
takes days! It's getting very painful to try and recover this computer! So,
I want to improve my ability to debug problems in the future... Any
suggestions on better ways to debug a remote server?

Is there a way to redirect the Linux console over the network like you can
to a serial port? My google searches keep leading me to "ethercons" for
FreeBSD which claims it is based on a Linux driver that I can't find any
information about. Is there something else that I haven't found? This would
help with interacting with the console to read error messages.

Is there a way to have the kernel reboot the system automatically if it
can't mount the root filesystem? I'm not 100% sure where the system is
hanging, but I think nash gives up when it can't find the filesystem and
just hangs. Can I specify that the kernel reboot on panic on the command
line? My setting in /etc/sysctl.conf doesn't get run because the root
filesystem isn't mounted yet!

Is there something I can use to trigger remote reboots? Like a wake-on-lan
packet that reboots the system when hung?

Is there a linux distro (preferably Fedora or one of its variants) that I
can boot a rescue session from using PXE? I'd like to build a setup with
grub, pxelinux, etc... (if possible) where it boots into the main OS and on
reboot, it will switch back-and-forth to a rescue distro. I just bought a
Linksys WRT54GL (only $50 on amazon!) and I plan on using openvpn to setup a
tunnel between this server and me and use the DHCP/TFTP server to serve up
PXE. Maybe with PXE I could serve one config for booting the real hard drive
and then another config when I want the rescue CD. This rescue distro would
need to bring up an ssh server with a password preset or a public key so I
could login remotely (if there is no way to redirect the console to the
network). I've considered, if no other solution is available, about rolling
a cusom Fedora spin with all the tools and configuration I need for this.

Thanks,
Ryan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://listman.redhat.com/archives/fedora-list/attachments/20081030/6187df87/attachment-0001.htm>