Skip to content

Instantly share code, notes, and snippets.

@seancribbs
Forked from bkerley/Gemfile
Last active December 15, 2015 00:29
Show Gist options
  • Save seancribbs/5173297 to your computer and use it in GitHub Desktop.
Save seancribbs/5173297 to your computer and use it in GitHub Desktop.
storing 10000 objects
done storing
Rehearsal ----------------------------------------------
individual 3.170000 0.250000 3.420000 ( 8.162000)
threaded/2 4.780000 0.350000 5.130000 ( 5.546000)
threaded/4 5.220000 0.410000 5.630000 ( 3.777000)
threaded/8 5.210000 0.430000 5.640000 ( 3.187000)
------------------------------------ total: 19.820000sec
user system total real
individual 2.640000 0.240000 2.880000 ( 7.898000)
threaded/2 3.530000 0.300000 3.830000 ( 5.325000)
threaded/4 5.240000 0.390000 5.630000 ( 3.709000)
threaded/8 4.850000 0.400000 5.250000 ( 3.146000)
require 'riak'
require 'benchmark'
HOSTS = (1..5).map do |n|
{host: '127.0.0.1', pb_port: "100#{n}7".to_i, http_port: "100#{n}8".to_i}
end
CORES = 8
NUMTHREADS = HOSTS.size
def threaded_get(client, bucket, keys, numthreads=NUMTHREADS)
slice_size = (keys.size / numthreads).ceil
threads = keys.each_slice(slice_size).map do |slice|
Thread.new do
slice.map do |k|
bucket[k]
end
end
end
threads.map {|t| t.join }
end
client = Riak::Client.new nodes: HOSTS, protocol: "pbc"
bucket = client.bucket 'threaded-get'
count = 10000
puts "storing #{count} objects"
keys = (1..count).map {|k| k.to_s }
keys.each do |k|
o = bucket.get_or_new k
o.content_type = 'text/plain'
o.raw_data = "the object_id of #{k} is #{k.object_id}"
o.store
end
puts "done storing"
Benchmark.bmbm do |x|
x.report("individual") do
keys.each {|k| bucket[k] }
end
x.report("threaded/2") do
threaded_get client, bucket, keys, 2
end
x.report("threaded/4") do
threaded_get client, bucket, keys, 4
end
x.report("threaded/8") do
threaded_get client, bucket, keys, 8
end
end
storing 10000 objects
done storing
Rehearsal ----------------------------------------------
individual 3.220000 0.170000 3.390000 ( 8.554107)
threaded/2 4.590000 0.400000 4.990000 ( 5.908789)
threaded/4 4.580000 0.590000 5.170000 ( 4.925002)
threaded/8 4.800000 0.610000 5.410000 ( 5.138573)
------------------------------------ total: 18.960000sec
user system total real
individual 2.980000 0.170000 3.150000 ( 8.352716)
threaded/2 4.510000 0.380000 4.890000 ( 5.832932)
threaded/4 4.610000 0.570000 5.180000 ( 4.939029)
threaded/8 4.600000 0.550000 5.150000 ( 4.882092)
storing 10000 objects
done storing
Rehearsal ----------------------------------------------
individual 3.250000 0.170000 3.420000 ( 8.702178)
threaded/2 4.830000 1.670000 6.500000 ( 6.946558)
threaded/4 4.760000 0.530000 5.290000 ( 5.053732)
threaded/8 4.580000 0.430000 5.010000 ( 4.812521)
------------------------------------ total: 20.220000sec
user system total real
individual 2.980000 0.170000 3.150000 ( 8.371760)
threaded/2 4.640000 1.640000 6.280000 ( 6.773202)
threaded/4 4.560000 0.530000 5.090000 ( 4.818344)
threaded/8 4.890000 0.440000 5.330000 ( 5.153690)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment