Skip to content

Instantly share code, notes, and snippets.

@bigsur0
bigsur0 / parallel_find_in_batches_start_end
Created September 23, 2011 16:53
Parallel gem and find_in_batches with start, end
num_processes = 2
batch_size = 1000
threads_per_process = 10
Parallel.map(0...num_processes, :in_processes => num_processes) do |i|·
User.connection.reconnect!
User.find_in_batches(:batch_size => batch_size,·
:start => (User.count/num_processes) * i,·
:conditions => ["id <= ?", (User.count/num_processes)*(i+1)]) do |batch|·
Parallel.map(batch, :in_threads => threads_per_process) do |record|