There are many options available for tuning ruby memory management: https://github.com/ruby/ruby/blob/trunk/gc.c#L7420-L7444
The one that gets the most attention is RUBY_GC_HEAP_GROWTH_FACTOR, which is the only one Heroku suggests adjusting: https://devcenter.heroku.com/articles/ruby-memory-use#gc-tuning
I've often wondered what the drawback was for setting RUBY_GC_HEAP_GROWTH_FACTOR quite low for a webapp. Wouldn't this only marginally affect performance while the app was initially booting up, and then therafter provide the most optimal memory size? So I set out to benchmark the behavior of different settings.
Assumption: since I'm benchmarking the time the memory manager takes to stop execution and allocate more memory, the code used for the benchmark can be anything which increases its memory usage as it runs. So diversity or realism is not needed.
- ruby 2.3.1p112
- macOS 10.11.6
- MacBook Pro (Retina, 15-inch, Mid 2014)
- 2.8 GHz Intel Core i7
puts ENV["RUBY_GC_HEAP_GROWTH_FACTOR"]
s = []
start = Time.now
25_000_000.times do |i|
print "[#{i/1_000_000}]" if 0 == i % 1_000_000
s << "abcdefabcdefabcdefabcdefabcdefabcdefabcdefabcdef"
end
finish = Time.now
puts
puts 'GC.stat[:heap_sorted_length]: ' + GC.stat[:heap_sorted_length].to_s
puts "seconds elapsed: " + (finish-start).to_s
sleep 100000000
➔ export RUBY_GC_HEAP_GROWTH_FACTOR= #default. i don't know what that is
➔ ruby code.rb
[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
GC.stat[:heap_sorted_length]: 82445
seconds elapsed: 6.431182
# memory, real memory, private memory: 1.78, 0.7, 0.6
➔ export RUBY_GC_HEAP_GROWTH_FACTOR=1.01
➔ ruby code.rb
1.01
[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
GC.stat[:heap_sorted_length]: 61783
seconds elapsed: 51.569609
# memory, real memory, private memory: 1.80, 1.51, 1.37
➔ export RUBY_GC_HEAP_GROWTH_FACTOR=1.1
➔ ruby code.rb
1.1
[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
GC.stat[:heap_sorted_length]: 67507
seconds elapsed: 9.097858
# memory, real memory, private memory: 1.80, 1.80, 1.67
➔ export RUBY_GC_HEAP_GROWTH_FACTOR=1.5
➔ ruby code.rb
1.5
[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
GC.stat[:heap_sorted_length]: 92055
seconds elapsed: 6.168111
# memory, real memory, private memory: 1.81, 1.81. 1.68
➔ export RUBY_GC_HEAP_GROWTH_FACTOR=2
➔ ruby code.rb
2
[0][1][2][3][4][5][6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23][24]
GC.stat[:heap_sorted_length]: 93538
seconds elapsed: 5.383548
# memory, real memory, private memory: 1.80, 1.80, 1.70
- number of slots allocated seems to correspond with what one would expect from increasing growth factor
- other than in the 1.01 case, time taken to stop code and allocate more memory is not significant, especially in the case of a web app booting up
- i don't know what the difference is between memory, real memory, and private memory (as reported by Activity Monitor). when i was a kid, we just had resident memory and virtual memory, and WE WERE HAPPY
- the default case is a wild outlier. If this were ruby 2.4 I would suspect it was using the RUBY_GC_HEAP_FREE_SLOTS_GOAL_RATIO algorithm (https://github.com/ruby/ruby/blob/trunk/gc.c#L7433-L7437). So I haven't the slightest guess what might be going on here.
got the idea to use heap_length here: https://samsaffron.com/archive/2013/11/22/demystifying-the-ruby-gc