Memory settings for Java JVM in Amazon Elastic Beanstalk

tomcat_memory.md

Elastic Beanstalk Java JVM Settings

Determining the right memory settings (MX & MS) for the JVM is not trivial. If you set memory too low then your machine will trash as it runs out and eventually crash. If you set it too high then other critical processes such as Apache or the OS itself may become memory starved and also cause crashes.

In general, I think it best to set the initial memory setting (MS) to be small, around 200M. The real variable we need to calculate is the limit we will place on JVM memory (MS).

In order to make this determination, we need to calculate a few things such as the memory that Apache and the Linux OS need to operate efficiently.

Apache Memory Needs

With default settings, Apache will need around 750M under heavy load. The calculations to make that determination follow.

The Apache Web Server footprint will basically be

Apache Web Server Footprint = MaxClients * (Max Memory per Child)

Default settings for Apache in Elastic Beanstalk are as follows

StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000

This is the base memory on an EC2 small instance for Apache when things are quiet. (Yes, this is a sweet bash one liner!!!!!!!!!!!!!!!!!)

ps -ylC httpd | awk '{x += $8;y += 1} END {print "Apache Memory Usage (MB): "x/1024; print "Average Proccess Size (MB): "x/((y-1)*1024)}'
Apache Memory Usage (MB): 23.0117
Average Proccess Size (MB): 2.55686

So if the child processes are at 3M (which I think they would be in a Tomcat type environment where Apache is just a forwarder) then we would have about 750M for Apache Web Server when it's under max load at 256 workers.

Amazon Linux Memory Footprint

A small EC2 Amazon instance seems to need about 300M – 400M of memory for the OS. See below.

top - 21:39:45 up  1:02,  1 user,  load average: 0.00, 0.01, 0.05
Tasks:  57 total,   1 running,  56 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1696516k total,   299948k used,  1396568k free,     9796k buffers
Swap:   917500k total,        0k used,   917500k free,   232616k cached

Recommendationd for Java JVM MX.

SMALL INSTANCES

1.7G (system memory) - 400M (system memory) - 750M (apache memory) = 550M for Tomcat and other
RECOMMENDATION IS 200M for MS and 500 for MX.

MEDIUM INSTANCES

3.75G (system memory) - 400M (system memory) - 750M (apache memory) = 2600M for Tomcat and other.
RECOMMENDATION IS 200M for MS and 2G for MS.

LARGE INSTANCES

7.5G (system memory) - 400M (system memory) - 750M (apache memory) = 2600M for Tomcat and other.
RECOMMENDATION IS 200M for MS and 5G for MS.

How do you configure ServerLimit (and other) in beanstalk ? I gues it's through .ebextensions, but how ?

There seems to be a few mistakes in the calculation for Large Instance. It should be
7.5G (system memory) - 400M (system memory) - 750M (apache memory) = 6350M for Tomcat and other.
RECOMMENDATION IS 200M for MS and 6G for MX.