Skip to content

Instantly share code, notes, and snippets.

@virtualstaticvoid
Last active May 22, 2019 17:51
Show Gist options
  • Save virtualstaticvoid/8705533 to your computer and use it in GitHub Desktop.
Save virtualstaticvoid/8705533 to your computer and use it in GitHub Desktop.
A better `find_each` and `find_in_batches` for ActiveRecord, which preserves the original order of the query
module ActiveRelationExtensions
def find_each_with_order(options = {})
find_in_batches_with_order(options) do |records|
records.each { |record| yield record }
end
end
# NOTE: any limit() on the query is overridden with the batch size
def find_in_batches_with_order(options = {})
options.assert_valid_keys(:batch_size)
relation = self
start = 0
batch_size = options.delete(:batch_size) || 1000
relation = relation.limit(batch_size)
records = relation.offset(start).to_a
while records.any?
records_size = records.size
yield records
break if records_size < batch_size
# get the next batch
start += batch_size
records = relation.offset(start + 1).to_a
end
end
end
ActiveRecord::Relation.send(:include, ActiveRelationExtensions)
module ActiveRecord
module Querying
delegate :find_each_with_order, :find_in_batches_with_order, :to => :scoped
end
end
@gregblass
Copy link

Thanks for this! Surprised to see that this isn't a part of Rails.

@gregblass
Copy link

This has a very serious bug that causes the method to drop 1 of them. It should just be offset(start) on line 30, not start + 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment