wiki outage

[Mike McGrath]

Mike McGrath wrote:
> The wiki is having an outage, we're working on it.  Should be up 
> soon.  We suspect our issues are because of mod_python.  The moin 
> developers suggest not using it.  So we'll go back to cgi (slow) and 
> deal with it (stable).

Ok, so that experiment lasted almost 5 minutes :-)  The box crapped out 
just after it hit a load around 200.  We had a whole team of sysadmins 
in #fedora-admin and did some things including enabling keep alive (no 
one even remembers why it was off) but also lowering the number of 
MaxRequestsPerChild significantly.  We were also working with the Moin 
developers to get this going. 

The moin developers suggested using mod_fastCGI, but it has an 
unfriendly license at present.   So we're back to mod_python for the 
moment (other things were suggested and will be tested if we're still 
seeing issues).  Things seem to be going well though, load is low,  the 
box doesn't seem to be swapping much.  We'll have to keep an eye on it 
and see how everything goes.

Anyone interested in any of the tests we ran and things we changed let 
me know.  We're just going to let it be for a while and if, in fact, we 
have fixed the issue then we'll migrate that back to the other app 
servers (its currently going proxy -> app server no balancing)

    -Mike