Skip to content

Instantly share code, notes, and snippets.

@newtonapple

newtonapple/bench_10000_1000000.rb

Last active August 29, 2015 14:21
Show Gist options
  • Save newtonapple/4094eb7700f27ce53530 to your computer and use it in GitHub Desktop.
Save newtonapple/4094eb7700f27ce53530 to your computer and use it in GitHub Desktop.
Download ZIP
Roshi Benchmarks on local iMac with default local settings (GOMAXPROCS=1)
Raw
bench_10000_1000000.rb
require './gen'
gen(10000, 1_000_000)
puts "generated data for: 10,000 users, 1 million events, 100 events per user."
puts "\ncurl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq ."
puts `curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .`
# note it will pass 10,000 user_ids on every select
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration`
__END__
generated data for: 10,000 users, 1 million events, 100 events per user.
curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .
{
"duration": "36.210438657s",
"inserted": 1000000
}
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration
"duration": "1.056505862s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration
"duration": "1.007086302s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration
"duration": "5.903297527s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration
"duration": "4.79454947s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration
"duration": "5.622054815s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration
"duration": "4.798538064s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration
"duration": "5.602761231s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration
"duration": "5.82867692s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration
"duration": "5.654054007s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration
"duration": "5.599620926s",
Raw
bench_1000_1000000.rb
require './gen'
gen(1000, 1_000_000)
puts "generated data for: 1000 users, 1 million events, 1000 events per user."
puts "\ncurl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq ."
puts `curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .`
# note it will pass 100 user_ids on every select
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration`
__END__
generated data for: 1000 users, 1 million events, 1000 events per user.
curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .
{
"duration": "33.340206527s",
"inserted": 1000000
}
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration
"duration": "106.282547ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration
"duration": "90.969225ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration
"duration": "1.170977009s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration
"duration": "953.511535ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration
"duration": "8.047272179s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration
"duration": "6.616333482s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration
"duration": "8.139027533s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration
"duration": "8.126881613s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration
"duration": "8.011347606s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration
"duration": "7.892007201s",
Raw
bench_100_1000000.rb
require './gen'
gen(100, 1_000_000)
puts "generated data for: 100 users, 1 million events, 10,000 events per user."
puts "\ncurl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq ."
puts `curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .`
# note it will pass 100 user_ids on every select
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration`
__END__
generated data for: 100 users, 1 million events, 10,000 events per user.
curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .
{
"duration": "29.458938125s",
"inserted": 1000000
}
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration
"duration": "9.516149ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration
"duration": "8.799324ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration
"duration": "89.694508ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration
"duration": "78.515764ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration
"duration": "1.17491457s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration
"duration": "1.003231389s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=500000' | jq . | grep duration
"duration": "5.517620224s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=500000' | jq . | grep duration
"duration": "5.397121376s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=900000' | jq . | grep duration
"duration": "5.44557301s",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=900000' | jq . | grep duration
"duration": "5.827904855s",
Raw
bench_50_50000.rb
require './gen'
gen(50, 50_000)
puts "generated data for: 50 users, 50,000 events, 1000 events per user."
puts "\ncurl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq ."
puts `curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .`
# note it will pass 50 user_ids on every select
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=25000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=25000' | jq . | grep duration`
puts "\ncurl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=45000' | jq . | grep duration"
puts `curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=45000' | jq . | grep duration`
__END__
generated data for: 50 users, 50,000 events, 1000 events per user.
curl -Ss -d@insert.json -XPOST 'http://localhost:6302' | jq .
{
"duration": "1.535495938s",
"inserted": 50000
}
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20' | jq . | grep duration
"duration": "4.988082ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=20' | jq . | grep duration
"duration": "5.007849ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=200' | jq . | grep duration
"duration": "56.652971ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=200' | jq . | grep duration
"duration": "51.42887ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000' | jq . | grep duration
"duration": "287.588113ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?limit=2000' | jq . | grep duration
"duration": "253.742298ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=20&offset=25000' | jq . | grep duration
"duration": "281.172212ms",
curl -Ss -d@select.json -XGET 'http://localhost:6302?coalesce=true&limit=2000&offset=45000' | jq . | grep duration
"duration": "285.317383ms",
Raw
gen.rb
require 'base64'
require 'json'
def gen(users, events)
user_ids = []
users.times{|i| user_ids << Base64.encode64(i.to_s) }
f = File.open('select.json', 'w')
JSON.dump(user_ids, f)
f.close
inserts = []
event_id = 0
events.times do |i|
event_id += i
user_id = user_ids[rand(users)]
inserts << {
key: user_id, score: event_id, member: Base64.encode64({user_id: user_id, event_id: event_id}.to_json)
}
end
f = File.open('insert.json', 'w')
JSON.dump(inserts, f)
f.close
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment