Restart TG apps for high mem-usage

[Mike McGrath]

Toshio Kuratomi wrote:
> Mike McGrath wrote:
>> Bill Nottingham wrote:
>>> Toshio Kuratomi (a.badger at gmail.com) said: 
>>>> Here's a short script to test our TG apps run via supervisor for 
>>>> excessive memory usage and restart them if necessary.  We could run 
>>>> this via cron in alternate hours on each app server.  Does this 
>>>> seem like a good or bad idea to people?
>>>>     
>>>
>>> It's a good idea if it's needed, but it's a bad idea that it is 
>>> needed. What's
>>> wrong with TG that it leads to this situation?
>>>   
>>
>> I was wondering this myself, I know smolt recently had some major 
>> changes to keep memory usage down.  Which TG apps are having this 
>> issue and how often?  I know MM uses a lot of memory but, AFAIK, it 
>> was determined that there's not much of a leak if there is one and 
>> that all of that memory is actually used.
>>
> Looks like smolt was upgraded just before Thanksgiving so it could be 
> that we've plugged the leaks we had to deal with that inspired me to 
> write this.  Would it be a good idea to have this in place anyways? 
> With it periodically checking, we would find out that we had problems 
> when cron emails us a notice that the script had to restart a process. 
> Without it, we'll be notified when nagios or a user tells us they're 
> getting timeouts.

I think its a good idea if for no other reason then allows us to more 
actively monitor this stuff, we'll get notified when the app restarts.  
+1 from me with the intention that, over time, we get fewer and fewer 
restarts.

    -Mike