Re: [K12OSN] cache running out of file descriptors (SOLVED)

[Mike Rambo <mrambo lsd k12 mi us>]

Eric Harrison wrote:
> 
> On Fri, 4 Apr 2003, Mike Rambo wrote:
> 
> >(among other things) Henrik Nordstrom wrote:
> >>
> >> You can also try the following settings to reduce the amount of
> >> filedescriptors used:
> >>
> >>   half_closed_clients off
> >>
> >> And if that does not help
> >>
> >>   server_persistent_connections off
> >>
> >> and finally if still a problem
> >>
> >>   client_persistent_connections off
> >>
> >
> >The first one didn't have any discernible effect. I tried the last two
> >this morning and have discovered it is a HUGE win. Not only have the
> >errors disappeared but access speed has increased very very noticeably.
> >According to cachemgr.cgi the number of requests per second we are
> >servicing has increased by fully one third. The difference in web
> >interactivity is amazing.
> >
> >
> 
> Interesting, I have not seen this mentioned as a performance improvement.
> In fact, I would have assumed that this would slow the cache down.
> I'll have to give it a spin ;-)
> 
> Here's the squid docs on these parameters:
> 
> http://squid.visolve.com/squid24s1/delaypool.htm#server_persistent_connections
> 
> -Eric
> 

Henrik's response regarding the downside of these settings is below. In
short, it probably helped me only because I already had a problem
running out of file descriptors - he says that ordinarily the default
settings are best. You asked me elsewhere about 

        cat /proc/sys/fs/file-max.

I remember checking this. It was at 8192 by default on the Mandrake 9
system. Squid was still only using 1024 file descriptors and was running
out. Once I knew where to look, cachemgr.cgi clearly showed that we
constantly using all available descriptors. Since changing those two
settings (and I haven't experimented yet to see which setting really
makes the difference - or whether both) I haven't seen more than 300 of
the 1024 available descriptors in use. I mentioned last Friday that
these changes had allowed us to service one-third more requests per
second. When I checked later in the day when our traffic peaked I found
that increase to actually be a little over 40% and had then become
limited by the bandwidth of our pipe to MSU. We had saturated that pipe
so I can't tell where the limit on the squid box might actually be.
Also, I can't remember whether I mentioned it last Friday but in
servicing the 40+% more requests per second the cpu utilization on the
squid box had dropped from the middle 80's to under 40 percent. That
lack of descriptors was really hurting us!

I'm not sure /proc/sys/fs/file-max is showing the wall I was hitting. It
appears more that 'ulimit -a' shows the open files limit I was up
against.

[mrambo squid mrambo]$ cat /proc/sys/fs/file-max
8192

[mrambo squid mrambo]$ ulimit -a
<snip>
open files                  1024
<snip>

I did increase that setting but it is apparently not enough to just
change that limit in the filesystem. From what I've read, Squid must
also be recompiled to pick up that setting and use the additional
descriptors.

Here is Henrik's response regarding those settings...

Henrik Nordstrom wrote:
> 
> fre 2003-04-04 klockan 18.20 skrev Mike Rambo:
> 
> 
> The first one has no negative effects for browsers. Basically only
> people trying to fetch HTTP via scrips using telnet or netcat see any
> difference from this one.. and when off Squid can immediately detect
> when a client aborts a request and tear down the connection. This makes
> a huge difference if there is problems reaching some frequently used web
> sites (or if you have a virus/worm which successfully makes it via the
> proxy hitting random targets..)
> 
> The other two are an optimization of the HTTP protocol to avoid
> repetition the TCP handshake overhead on each request by keeping
> connections open. But as you may have discovered, Squid is not very
> clever in getting rid of unused such connections before they expire when
> running short of filedescriptors (Bug #571).. Normally performance is
> slightly increased by having these on (maybe 10% reduction in latency),
> but if you run short of filedescriptors then the direct opposite effect
> is seen due to the filedescriptor shortage making Squid no longer accept
> new connections..
> 
> Regards
> Henrik
> 


-- 
Mike Rambo
mrambo lsd k12 mi us