debug system freeze

[stan]

On Thu, 03 Sep 2009 21:36:39 -0400
brian <fedora at logi.ca> wrote:

> Every couple of days, for the past couple of weeks, my computer locks
> up on me. I can't drop out of X; the only thing to do is hold down
> the power button to shut it down. Can anyone suggest how to debug
> this?
> 

My thought is that it would take some serious effort to debug this.
Recompile the kernel and turn on as many of the debugging options in
kernel hacking as possible.  Run the modified kernel in a virtual
machine or separate install and use lots of logging that is short term
(you don't need logs from hours ago, 5 minutes of logs is probably
plenty).  When it hangs, boot into another system to examine the logs of
the frozen system.  The logging might prevent the problem from
happening, depending on what it is, but you have a chance of catching it
in the act.

When this was happening to me, I thought it behaved like the IP stack
was being corrupted.  That means that the system is frozen, because
there was no next instruction.  Dead, defunct, deceased.  Only post
mortem examination possible.

I've seen an option in the kernel to allow a second kernel to be built
and swap in when the first crashes in order to do the post mortem, but
I'm not sure that would work in this case, and in any case I haven't
actually used it.