Skip to content

Instantly share code, notes, and snippets.

@maccman
Created September 22, 2014 01:34
Show Gist options
  • Save maccman/81d17c6b2cd962dbf228 to your computer and use it in GitHub Desktop.
Save maccman/81d17c6b2cd962dbf228 to your computer and use it in GitHub Desktop.
Batch Paginate in SQL
module Sequel
module Plugins
module Batch
module DatasetMethods
def batch_page(page_size = 2_000)
last_record = order(:iid.asc).first
last_iid = last_record && last_record.iid || 0
last_iid -= 1
loop do
result = order(:iid.asc)
.where { iid > last_iid }
.limit(page_size)
.all
break if result.empty?
yield result
last_iid = result.last.iid
end
self
end
def batch_row(page_size = 2_000)
return to_enum(:batch_row, page_size) unless block_given?
batch_page(page_size) do |records|
records.each do |record|
yield(record)
end
end
self
end
end
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment