This was going to go into a blog post, but was removed after discussion around it being a Bad Thing for customers to do.
To stop applications that are prone to bloating quickly exhausting all the memory on an application instance, we have a script called passenger_monitor that checks for runaway Passenger processes every minute via a cron job. By default, this script is going to look for Passenger workers that are using over 215MB of memory and kill them. Passenger will restart them when needed. We tune the number of workers that run by default on your application instances, based on the memory specifications so that the memory is sensibly utilized. However, if your application legitimately requires more memory than this (i.e. it isn’t bloating) then it may be advantageous for you to reduce the worker count, and allow the lower number of workers to use more memory.
Doing this involves two areas of customization; modifying the Nginx configuration of passenger_max_pool_size and modifying the crontabs on the application instances to increase the memory limit used with passenger_monitor.
The configuration of the first part just involves using a template to modify the /etc/nginx/stack.conf file, however the modification of the crons is slightly more interesting to look at and is accomplished like so:
https://gist.github.com/1272379
Chef is able to find cron jobs via name, so what we do here is to remove the original cron job, then just recreate it with our modified call to the passenger_monitor script. To find out the Chef name of any cron job, you can view the output of the crontab -l command. In this particular instance, you would need to view root’s crontab listing by running:
sudo crontab -uroot -l
Here’s an example cookbook that carries out the discussed changes:
Please note: reducing the max pool size is not recommended for multi-app environments, as the workers have to be spread across your different applications. Dropping them in number could result in the spawner having to kill off workers for one app, to spin them up for another, causing increased load on the instance and a likely increase in server errors.