Skip to content

Instantly share code, notes, and snippets.

@abhchand
Last active October 2, 2017 14:06
Show Gist options
  • Save abhchand/50946bfd95c6b676c4976ed83cc13f00 to your computer and use it in GitHub Desktop.
Save abhchand/50946bfd95c6b676c4976ed83cc13f00 to your computer and use it in GitHub Desktop.
Blog Post: Speeding up ActiveRecord iterations on larger datasets
N = 1_000
Benchmark.bm do |x|
x.report("all columns") do
N.times do
User.find_by_sql("SELECT * FROM users LIMIT 1000;")
end
end
x.report("one column") do
N.times do
User.find_by_sql("SELECT email FROM users LIMIT 1000;")
end
end
end
user system total real
all columns 29.530000 2.280000 31.810000 ( 34.459085)
one column 19.110000 1.040000 20.150000 ( 21.719567)
ActiveRecord::Base.connection.execute("SELECT email FROM users;").each do |record|
email = record["email"]
# ...
end
User.select(:email).find_each do |user|
# ...
end
require 'benchmark'
N = 1_000
conn = ActiveRecord::Base.connection
Benchmark.bm do |x|
# Include a random OFFSET in below queries so cached results are not returned
x.report("all columns") do
N.times do
conn.execute("SELECT * FROM users LIMIT 1000 OFFSET #{(1000*rand).round};")
end
end
x.report("one column") do
N.times do
conn.execute("SELECT email FROM users LIMIT 1000 OFFSET #{(1000*rand).round};")
end
end
end
user system total real
all columns 1.920000 2.510000 4.430000 ( 16.687199)
one column 0.370000 0.260000 0.630000 ( 2.804193)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment