[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]

Re: Restart TG apps for high mem-usage

Mike McGrath wrote:
Toshio Kuratomi wrote:
Mike McGrath wrote:
Bill Nottingham wrote:
Toshio Kuratomi (a badger gmail com) said:
Here's a short script to test our TG apps run via supervisor for excessive memory usage and restart them if necessary. We could run this via cron in alternate hours on each app server. Does this seem like a good or bad idea to people?

It's a good idea if it's needed, but it's a bad idea that it is needed. What's
wrong with TG that it leads to this situation?

I was wondering this myself, I know smolt recently had some major changes to keep memory usage down. Which TG apps are having this issue and how often? I know MM uses a lot of memory but, AFAIK, it was determined that there's not much of a leak if there is one and that all of that memory is actually used.

Looks like smolt was upgraded just before Thanksgiving so it could be that we've plugged the leaks we had to deal with that inspired me to write this. Would it be a good idea to have this in place anyways? With it periodically checking, we would find out that we had problems when cron emails us a notice that the script had to restart a process. Without it, we'll be notified when nagios or a user tells us they're getting timeouts.

I think its a good idea if for no other reason then allows us to more actively monitor this stuff, we'll get notified when the app restarts. +1 from me with the intention that, over time, we get fewer and fewer restarts.

Cool.  I'll check it in and set up a cron job.

One further piece of information since I have output from testing this on app3 yesterday:

AppName        Uptime  RSS 11/25   RSS 11/26
mirrormanager  2d4h    714336      962268
packagedb      --- restarted 13h ago --
smolt          5d3h    299556      299556
transifex      5d3h    42744       42768


[Date Prev][Date Next]   [Thread Prev][Thread Next]   [Thread Index] [Date Index] [Author Index]