Last active
October 2, 2017 14:06
-
-
Save abhchand/50946bfd95c6b676c4976ed83cc13f00 to your computer and use it in GitHub Desktop.
Blog Post: Speeding up ActiveRecord iterations on larger datasets
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
N = 1_000 | |
Benchmark.bm do |x| | |
x.report("all columns") do | |
N.times do | |
User.find_by_sql("SELECT * FROM users LIMIT 1000;") | |
end | |
end | |
x.report("one column") do | |
N.times do | |
User.find_by_sql("SELECT email FROM users LIMIT 1000;") | |
end | |
end | |
end | |
user system total real | |
all columns 29.530000 2.280000 31.810000 ( 34.459085) | |
one column 19.110000 1.040000 20.150000 ( 21.719567) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ActiveRecord::Base.connection.execute("SELECT email FROM users;").each do |record| | |
email = record["email"] | |
# ... | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
User.select(:email).find_each do |user| | |
# ... | |
end |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
require 'benchmark' | |
N = 1_000 | |
conn = ActiveRecord::Base.connection | |
Benchmark.bm do |x| | |
# Include a random OFFSET in below queries so cached results are not returned | |
x.report("all columns") do | |
N.times do | |
conn.execute("SELECT * FROM users LIMIT 1000 OFFSET #{(1000*rand).round};") | |
end | |
end | |
x.report("one column") do | |
N.times do | |
conn.execute("SELECT email FROM users LIMIT 1000 OFFSET #{(1000*rand).round};") | |
end | |
end | |
end | |
user system total real | |
all columns 1.920000 2.510000 4.430000 ( 16.687199) | |
one column 0.370000 0.260000 0.630000 ( 2.804193) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment