Skip to content

Instantly share code, notes, and snippets.

@quezacoatl
Last active May 10, 2016 02:11
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save quezacoatl/7657854f371edcb5d8e6 to your computer and use it in GitHub Desktop.
Save quezacoatl/7657854f371edcb5d8e6 to your computer and use it in GitHub Desktop.
Ruby 2.2.X AWS SDK memory leak
source 'https://rubygems.org'
gem 'nokogiri'
gem 'aws-sdk'
gem 'fake_sqs'
gem 'newrelic_rpm'
GEM
remote: https://rubygems.org/
specs:
aws-sdk (2.0.29)
aws-sdk-resources (= 2.0.29)
aws-sdk-core (2.0.29)
builder (~> 3.0)
jmespath (~> 1.0)
multi_json (~> 1.0)
multi_xml (~> 0.5)
aws-sdk-resources (2.0.29)
aws-sdk-core (= 2.0.29)
builder (3.2.2)
fake_sqs (0.1.0)
builder
sinatra
jmespath (1.0.2)
multi_json (~> 1.0)
mini_portile (0.6.2)
multi_json (1.11.0)
multi_xml (0.5.5)
newrelic_rpm (3.10.0.279)
nokogiri (1.6.6.2)
mini_portile (~> 0.6.0)
rack (1.6.0)
rack-protection (1.5.3)
rack
sinatra (1.4.5)
rack (~> 1.4)
rack-protection (~> 1.4)
tilt (~> 1.3, >= 1.3.4)
tilt (1.4.1)
PLATFORMS
ruby
DEPENDENCIES
aws-sdk
fake_sqs
newrelic_rpm
nokogiri
$ rvm use --default ruby-2.2.1
Using /home/quezacoatl/.rvm/gems/ruby-2.2.1
$ ruby sqs_leak_test.rb
31.98828125
32.421875
32.609375
32.6953125
32.765625
32.828125
32.83984375
32.875
32.921875
32.9296875
32.93359375
32.9375
32.96484375
32.97265625
33.18359375
33.234375
33.25
33.25390625
33.3359375
33.2734375
33.296875
33.3203125
33.328125
33.328125
33.3515625
33.359375
33.3671875
33.38671875
33.4140625
33.58984375
33.58984375
33.69921875
33.765625
33.875
33.9765625
34.03125
34.05859375
34.0859375
34.10546875
34.1328125
34.14453125
34.30078125
34.26171875
34.28515625
34.3046875
34.30859375
34.4375
34.47265625
34.484375
34.52734375
34.58203125
34.6171875
34.62890625
34.63671875
34.67578125
34.69140625
34.6953125
34.69921875
34.69921875
34.69921875
34.7578125
34.83203125
34.859375
34.921875
34.98046875
35.0078125
35.0078125
35.26171875
35.34765625
35.375
35.37890625
35.37890625
35.4296875
35.4296875
35.43359375
35.453125
35.4609375
35.46875
35.5625
35.59375
35.62890625
35.6484375
35.80078125
35.84765625
35.87109375
35.8984375
35.90625
35.92578125
35.94921875
35.953125
35.96875
35.98046875
35.9921875
36.01953125
36.2265625
36.2265625
36.2265625
36.30859375
36.3125
36.33203125
36.41796875
36.4453125
36.56640625
36.62890625
36.640625
36.68359375
36.67578125
36.8046875
36.828125
36.8828125
36.875
36.91015625
37.02734375
37.04296875
37.04296875
37.04296875
37.34765625
37.34765625
37.4296875
37.44921875
37.44921875
37.47265625
37.5703125
37.5703125
37.61328125
37.640625
37.67578125
37.6796875
37.78515625
37.8359375
37.87890625
37.92578125
37.99609375
38.04296875
38.125
38.1875
38.1796875
38.18359375
38.4140625
38.42578125
38.4453125
38.56640625
38.609375
38.625
38.66796875
38.671875
38.671875
38.70703125
38.70703125
38.70703125
38.94140625
38.953125
38.9765625
39.01953125
39.03515625
39.08203125
39.21875
39.21875
39.2890625
39.3046875
39.31640625
39.328125
39.36328125
39.36328125
39.55078125
39.55078125
39.62109375
39.6484375
39.69140625
39.69921875
39.7265625
39.7578125
39.7734375
39.84765625
39.8828125
40.0078125
40.0078125
40.09765625
40.1015625
40.13671875
40.1953125
40.34765625
40.2734375
40.328125
40.37890625
40.421875
40.53125
40.59765625
40.62890625
40.64453125
40.68359375
40.703125
40.71875
40.7421875
40.80078125
40.8046875
40.8125
40.94140625
40.9765625
40.98828125
41.0390625
41.13671875
41.2109375
41.25390625
41.296875
41.328125
41.33203125
41.40625
41.44921875
41.55078125
41.59375
41.63671875
41.66015625
41.71484375
41.76171875
41.80859375
41.87109375
.
.
.
$ rvm use ruby-2.2.0
Using /home/quezacoatl/.rvm/gems/ruby-2.2.0
$ ruby sqs_leak_test.rb
38.6953125
39.984375
40.2421875
40.5
40.859375
40.9765625
41.05078125
41.06640625
41.16796875
41.21484375
41.265625
41.3125
41.37109375
41.37109375
41.46484375
41.52734375
41.52734375
41.61328125
41.63671875
41.65234375
41.65234375
41.65234375
41.66015625
41.66796875
41.66796875
41.66796875
41.66796875
41.671875
41.67578125
41.67578125
41.67578125
41.67578125
41.6796875
41.6796875
41.6796875
41.6953125
41.6953125
41.6953125
41.703125
41.71484375
41.71484375
41.72265625
41.734375
41.734375
41.734375
41.734375
41.734375
41.734375
41.734375
41.73828125
41.74609375
41.74609375
41.75
42.05078125
42.05859375
42.26171875
42.30078125
42.3203125
42.34375
42.3203125
42.4375
42.47265625
42.47265625
42.58203125
42.61328125
42.61328125
42.69921875
42.7421875
42.76953125
42.81640625
42.81640625
42.88671875
42.88671875
42.97265625
43.015625
43.05078125
43.1015625
43.1015625
43.21875
43.25390625
43.29296875
43.34375
43.34375
43.41796875
43.41796875
43.4921875
43.546875
43.58203125
43.62890625
43.68359375
43.7265625
43.76953125
43.796875
43.8359375
43.86328125
43.9375
43.984375
44.04296875
44.08203125
44.08203125
44.2421875
44.2421875
44.40234375
44.40234375
44.5703125
44.57421875
44.75
44.8046875
44.8046875
44.890625
44.890625
45.0703125
45.12890625
45.12890625
45.23046875
45.26953125
45.26953125
45.34765625
45.38671875
45.421875
45.46875
45.5078125
45.54296875
45.59765625
45.62890625
45.671875
45.671875
45.73828125
45.7734375
45.82421875
45.82421875
45.89453125
45.9296875
45.96484375
45.96484375
46.04296875
46.0859375
46.1171875
46.16015625
46.16015625
46.2578125
46.30078125
46.30078125
46.4296875
46.4765625
46.53515625
46.58203125
46.6171875
46.67578125
46.70703125
46.7578125
46.7890625
46.84765625
46.84765625
46.921875
46.9609375
46.99609375
46.99609375
47.06640625
47.06640625
47.171875
47.22265625
47.22265625
47.43359375
47.47265625
47.4765625
47.53515625
47.57421875
47.57421875
47.6484375
47.68359375
47.71484375
47.75
47.79296875
47.828125
47.828125
47.90625
47.94140625
47.9765625
47.9765625
48.05859375
48.09765625
48.09765625
48.1640625
48.1640625
48.23046875
48.27734375
48.27734375
48.35546875
48.38671875
48.38671875
48.46484375
48.515625
48.55859375
48.59765625
48.62890625
48.671875
48.70703125
48.70703125
48.78515625
48.78515625
48.859375
48.859375
48.93359375
48.96875
48.96875
48.96875
48.96875
49.12890625
49.12890625
49.19140625
49.19140625
49.265625
49.30859375
49.35546875
49.35546875
49.41015625
49.453125
49.49609375
49.52734375
49.52734375
49.6015625
49.63671875
49.63671875
49.7109375
49.76171875
49.80078125
49.83984375
49.875
49.875
49.953125
49.953125
50.02734375
50.02734375
50.1015625
50.1015625
50.1796875
50.1796875
50.25390625
50.25390625
50.328125
50.328125
$ rvm use ruby-2.1.2
Using /home/quezacoatl/.rvm/gems/ruby-2.1.2
$ ruby sqs_leak_test.rb
36.80859375
37.06640625
37.1484375
37.62890625
37.6484375
37.70703125
37.734375
37.734375
37.734375
37.98828125
37.98828125
37.98828125
37.98828125
37.953125
37.95703125
37.95703125
37.95703125
37.95703125
37.95703125
37.95703125
37.95703125
37.95703125
37.95703125
37.99609375
37.9921875
37.9453125
37.8359375
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.8828125
37.94921875
37.953125
37.953125
37.953125
37.953125
37.953125
37.953125
37.953125
37.953125
37.953125
38.0546875
38.05078125
38.05078125
38.04296875
38.03125
38.03125
38.03125
38.02734375
38.02734375
38.0
37.98046875
37.98046875
37.98046875
37.98046875
37.98046875
37.98046875
37.98046875
37.98046875
37.96875
37.97265625
require 'bundler/setup'
require 'aws-sdk'
require 'newrelic_rpm'
Aws.config = {
access_key_id: 'fake access key',
secret_access_key: 'fake secret key',
}
sqs = Aws::SQS::Client.new(endpoint: 'http://localhost:4568')
begin
sqs.create_queue(queue_name: 'test')
rescue Aws::SQS::Errors::QueueNameExists => e
# ignore if queue already exists
end
loop do
for i in 0..100
sqs.receive_message(
queue_url: 'http://0.0.0.0:4568/test',
max_number_of_messages: 10,
wait_time_seconds: 0,
)
end
GC.start
puts NewRelic::Agent::Samplers::MemorySampler.new.sampler.get_sample
end
@quezacoatl
Copy link
Author

I've used fake_sqs here, but a normal SQS queue works perfectly fine.

@ko1
Copy link

ko1 commented Mar 6, 2015

I'm not sure about these middleware.
How to make server http://0.0.0.0:4568?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment