Skip to content

Instantly share code, notes, and snippets.

@mtyaka
Created March 9, 2012 16:47
Show Gist options
  • Save mtyaka/2007446 to your computer and use it in GitHub Desktop.
Save mtyaka/2007446 to your computer and use it in GitHub Desktop.
Batch insert images into ES index
website = Website.first
es = ElasticSearch.new('localhost:9200', :index => "tags", :type => "image")
count = 0
images = website.images.includes(:meta_tags).where("copied_image_id is ? and deleted = ?", nil, 0);
images.each_slice(1000) do |images|
puts count if count % 10 == 0
es.bulk do |client|
images.each do |image|
tags = image.meta_tags.map {|t| t.category_and_name }
client.index({:id => image.id, :tags => tags, :code => image.code}, :id => image.id)
count = count + 1
end
end
end
@danielcremer
Copy link

Great thanks. It's a clean solution. I'll try it out with a lot of images.
If this works we can try putting the images into a beanstalkd queue to batch the inserts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment