Skip to content

Instantly share code, notes, and snippets.

@kidpollo
Created November 2, 2010 19:51
Show Gist options
  • Save kidpollo/660183 to your computer and use it in GitHub Desktop.
Save kidpollo/660183 to your computer and use it in GitHub Desktop.
module Processors
class ImagesProcessor
def initialize(queue, image_cache_options = {})
@queue = queue || {}
@image_cache = Processors::ImageCache.new(image_cache_options)
@logger = CSV_PROCESSOR_LOG
end
def process
@logger.info("Image queue size: #{@queue.size}")
@queue.each do |item|
@logger.info("Processing: #{item[:url]} Mem: #{`ps -o rss= -p #{$$}`.to_i}")
process_queue_item(item)
end
end
private
def process_queue_item(item)
path = @image_cache.find(item[:url])
if path
attributes = item.reject do |key, value|
key == :url || key == :find_by || key == :klass
end
klass = item[:klass]
image = if item[:find_by] == :owner
klass.find_by_owner_id_and_owner_type(item[:owner_id], item[:owner_type])
else
klass.find_by_id(item[:id])
end
image = klass.new unless image
image.attributes = attributes
if image.last_scraped_url != item[:url]
File.open(path) do |file|
image.attachment = file
end
image.last_scraped_url = item[:url]
image.save
end
image = nil
GC.start
end
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment