Apache webserver outage - need help with forensics

[Nigel Wade]

Bob Brennan wrote:
> I have a server which went completely unresponsive today on port 80
> for 20 minutes and would appreciate any pointers as to what might have
> happened.
> 
> A bit of background:
> * FC3, Up2Date
> * The Apache webserver serves a dozen virtual websites
> * Sendmail + Dovecot + Squirrelmail for all sites
> * Spamassasin recently activated (yesterday)
> 
> The problem + observations:
> * All websites were inaccessible from 14:00 gmt to 14:20 today
> * The mailserver was running and responsive during that time
> * FTP was running and responsive during that time
> * telnet theServer.com 80 timed out with no connection during that time
> 
> What I checked:
> * all access_log and error_log for all sites - showed 5 users using
> the sites at the time but nothing unusual
> * no evidence of a DOS attack (that I could see)
> * no records of anything unusual in system logs
> * no accesses or errors in any of the http logs during that time
> 
> Thankfully the webserver came back as if by magic after 20 minutes and
> was immediately responsive.
> 
> Any insights into anything else I can check? Needless to say an
> embarassing incident for a webmaster who whould like to prevent it
> happening again.
> 
> Thanks in advance,
> bob
> 

Maybe either a deliberate or unintentional DoS attack.

How many clients is your server configured to handle simultaneously? Maybe 
there was a problem, or some deliberate attack, which meant the established 
clients communications stuck and no new client connections could be accepted.

Did you have netstat output to show what connections were established to 
port 80 at the time?

-- 
Nigel Wade, System Administrator, Space Plasma Physics Group,
             University of Leicester, Leicester, LE1 7RH, UK
E-mail :    nmw at ion.le.ac.uk
Phone :     +44 (0)116 2523548, Fax : +44 (0)116 2523555