Skip to content

Instantly share code, notes, and snippets.

@Chocksy
Last active September 5, 2023 06:46
Show Gist options
  • Save Chocksy/6efa4833989f51e46fca to your computer and use it in GitHub Desktop.
Save Chocksy/6efa4833989f51e46fca to your computer and use it in GitHub Desktop.
Searchkick reindex with resume
namespace :searchkick do
desc "maybe reindex without running out of memory?"
task :altindex, [:start_index, :batch_size, :new_index_name] => :environment do |t, args|
ActiveRecord::Base.logger = Logger.new(STDOUT)
count = Item.count
start = args[:start_index].to_i || 0
batch_size = args[:batch_size].to_i || 1000
# we create a new index or take one already present
if args[:new_index_name]
new_index = Searchkick::Index.new(args[:new_index_name])
puts "we took a existing index"
else
new_index = Item.searchkick_index.create_index
puts "the new created index name is: #{new_index.name}"
end
puts "Starting reindex at #{start} with batch size of #{batch_size}..."
Item.find_in_batches(:start=> start, :batch_size=> batch_size).with_index do |batch, index|
puts "Processing batch ##{index}; total #{index*batch_size+batch_size} items"
before = Time.now
new_index.import(batch)
time = Time.now - before
puts "Processed #{batch_size} items in #{time} secs"
end
Item.searchkick_index.swap(new_index.name)
end
end
@vinchi777
Copy link

How do we know where to start? or how did you get the start value?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment