Skip to content

Instantly share code, notes, and snippets.

@CGA1123
Last active June 9, 2020 18:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save CGA1123/143ca9376397c0f09c96e39313479281 to your computer and use it in GitHub Desktop.
Save CGA1123/143ca9376397c0f09c96e39313479281 to your computer and use it in GitHub Desktop.
class BackfillJob < ApplicationJob
queue_as :backfill
def perform(file_num:, max_file_num:, batch_size: 500)
fetch_from_s3(file_num).each_slice(batch_size) do |ids|
FactoryOrderQuote.where(id: ids).update_all(status: :not_renewable)
end
return if file_num >= max_file_num
BackfillJob.perform_later(
file_num: file_num + 1,
max_file_num: max_file_num,
batch_size: batch_size
)
end
def fetch_from_s3(file_number)
s3_bucket
.object(file_name(file_number))
.get
.body
.read
.then(&CSV.method(:parse))
.flatten
.map(&:to_i)
end
def file_name(file_number)
country = L10n.country
"#{country}/factory_order_quotes-#{country}-split-#{file_number.to_s.rjust(5, '0')}.csv"
end
def s3_bucket
Aws::S3::Resource
.new(region: 'my-aws-region')
.bucket('my-bucket')
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment