tux on 2.4.27 kernel and referrer checking

Thu Oct 28 23:55:04 UTC 2004

On Thu, Oct 28, 2004 at 06:18:16PM -0500, William Lovaton scribbled:
> El jue, 28-10-2004 a las 17:37, Marek Habersack escribió:
> > > I haven't use FC2 in desktops that much but I love what I see, feature
> > > wise and stability wise.  In fact, I'm dying to use FC3 in my
> > > workstation (home and office) ASAP.
> > I cannot be a partner for discussion here since I don't use any of the
> > RedHat systems anymore, but I do think it's not a good idea to use the same
> > kernel for both desktop and server (think about the i/o scheduler for
> > instance, or the filesystems you use).
> 
> Agree, but the new process scheduler is a lot smarter and it does a
> really good job.
Indeed

> > > > As for 2.4, it has serious problems with
> > > > the VM under stress (heavy webserver load with php/sql activity) to the
> > > > point when an OOM kills not only the process(es) but also the machine.
> > > 
> > > OOM?? May be you have some serious issues with your lack of memory or
> > > software leaking a lot.
> > Unlikely. Even then, a user application should NOT be able to kill the
> > kernel. What we see is the treaded 0 order allocation failed, apache
> > (because it's apache in 99% of case) gets killed and the kernel freezes. No
> > oops, nothing in the logs. It's not a unit issue, it happened on several
> > machines (ranging from P3 to P4 systems) with at least 0.5G of RAM and twice
> > as much of swap.
> 
> AFAICT, your problem is lack of RAM.  Our system is limited to 340
> processes with 2GB of RAM.  It use the whole RAM, with about 800MB of
> cache so it never swaps.  If you start to swap, then you need more RAM
> (or fix the programs).
Hm, I wouldn't take it as a rule of the thumb. In general, if the software
is well written (i.e. it doesn't run away with memory usage or doesn't
generate unnecessary load) then swapping isn't bad, in fact, it's probably a
good idea to let the mostly dormant processes swap themselves out. But in
any way, the kernel should not die even if the user apps do the most stupid
things - that's why I cannot wait to use strict commit in 2.6 in production.
Tux helped a lot with the load, but it cannot help when PHP is used, alas.
In our case it's, as I've said, mostly bad programming on the clients part,
but still and again... it should not be possible for a runaway app to crash
the kernel.

> > > In my case high load means 115 requests/sec according to apache
> > > access_log 70% of them are static content, the rest is dynamic (PHP).  I
> > > have seen peaks of 320 req/sec.  This is why TUX is so great.
> > > 
> > > The load average is between 40 and 80 during peak times without tux. 
> > > The server gets 4.5 millions of requests per day.
> > Some of our machines all of a sudden get load average over 300, one of them
> > had load of over 900 last week. Surely, most of the cases can be attributed
> > to bad programming (php, perl scripts that do weird things), but that should
> > not crash the kernel. At least I hope so. We tried resource limits, but on a
> > reasonably busy machine it's rather hard to come up with sensible resource
> > limits for over 800 apache processes. If you give each of them 1/3 of the
> > RAM to use as the maximum, it's enough that 6 of them run away and the
> > machine dies. They are dedicated severs handling an awful lot of traffic, so
> > limiting the number of apache processes below 512 makes no real sense.
> > vservers might be a good promise for enforcing resource limits perhaps, but
> > that only with 2.6, so not for us yet.
> 
> Mmmm... load avg of 300, 800... that's insane.  You just need a bigger
> CPU this is just a matter of MHz.  With such a load you can't even type
> on a terminal.
Definitely so :>, but throwing in hardware is not a cure for bad software
(kernel not being the bad one here). That way you're only avoiding finding
the real reason of the load.

> 800 load avg means that there are 800 processes waiting to be run, but
> they are not running because the CPU is taken by another process.  The
> processes are just fighting to get a little slice of CPU.
A lot of those processes are in the D state, i.e. waiting for I/O. So they
are not really 'running', but are being counted in the load average.

> And 800 processes?? How much RAM do you have? what are the technical
> specs of the server?
P3/800 w 512 of RAM, for instance. And, trust me, with correctly written
software such machine can do a LOT.

> BTW, if you upgrade the CPU you will need less RAM.
as in that processes will execute their stuff faster and return the memory
to the system? I don't think the two things are related in that way, if
processes are executed faster, there will be more of them running in a unit
of time, that's it. The memory usage will either remain the same in effect
or increase.

> > > > > The only problem I have is TUX (not using it right now) and that's why
> > > > > I'm subscribed to this list.  Anyway TUX is not present in the official
> > > > > kernel anymore.
> > > > What is the problem with TUX?
> > > 
> > > http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=125091
> > Sounds as awful as the bug Ingo fixed sometime ago for us - but we had a
> > problem with the user space modules for Tux.
> 
> It should be fixed by now (I hope) but I can't test it right now because
> the updated kernel is for the new FC3.  So I just have to wait for the
> appropiate update for FC2.
Do you always run stock vendor kernels? (just out of curiosity)

regards,

marek
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://listman.redhat.com/archives/tux-list/attachments/20041029/ade106ee/attachment.sig>