Unicorn memory usage improvement with Ruby 2.0.0

gistfile1.md

Unicorn memory usage improvement with Ruby 2.0.0

Here's a preliminary experiment to see how much memory is saved with the new copy-on-write friendly (bitmap marking) GC.

Calculated by memstats.rb https://gist.github.com/kenn/5105061 on Debian x86_64.

Master process:

# ./memstats.rb 20547
Memory Summary:
  private_clean                 448 kB
  private_dirty              24,492 kB
  pss                        28,278 kB
  rss                        73,144 kB
  shared_clean                4,520 kB
  shared_dirty               43,684 kB
  size                      184,120 kB
  swap                            0 kB

Worker process:

# ./memstats.rb 20554
Memory Summary:
  private_clean                   0 kB
  private_dirty              25,012 kB
  pss                        27,695 kB
  rss                        71,216 kB
  shared_clean                2,648 kB
  shared_dirty               43,556 kB
  size                      255,504 kB
  swap                            0 kB

rss represents the physical memory that is actually used, and it's comprised of private_clean + private_dirty + shared_clean + shared_dirty.

For the worker process, among 71,216 KB of physical memory, 46,204 KB is shared and 25,012 KB is private, that is, 65% of its memory is shared between workers and the master.

In other words, the footprint of a unicorn worker on this system is only 25MB instead of 70MB. I think it's a big deal!

Was wondering how MRI 2.0's GC would deal with being CoW friendly and Unicorn when preload_app is set to true. Seems like you've already done the research. This, combined with Heroku's x2 dyno (double ram and double cpu share) might be very interesting.

Thanks for sharing!

Odd to reply here, but I've been having trouble that over time, the master processes's memory get swapped out, so new forks are not using the shared memory in a very nice manner...

You will gain the same memory savings with Phusion Passenger. In fact, Phusion Passenger did the same thing in combination with Ruby Enterprise Edition a couple of years ago. It's great that the functionality has finally found its way to upstream Ruby.