Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@slbug

slbug/Gemfile

Created February 25, 2016 14:50
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save slbug/7b4d791c80ebde4361b9 to your computer and use it in GitHub Desktop.
Save slbug/7b4d791c80ebde4361b9 to your computer and use it in GitHub Desktop.
Download ZIP
Deadlock using sidekiq & fog
Raw
README
run sidekiq:
DEBUG=true EXCON_DEBUG=true key=aws_key secret=aws_secret bucket=your_bucket sidekiq -C ./sidekiq.yml -r ./s3_worker.rb
run irb:
irb -r ./s3_worker.rb
run S3Worker.perform_async in irb
Raw
Gemfile
ruby '2.3.0'
source 'http://rubygems.org'
gem 'sidekiq'
gem 'fog'
gem 'mime-types'
Raw
s3_worker.rb
require 'sidekiq'
require 'stringio'
require 'zlib'
require 'fog'
Sidekiq.configure_server do |config|
config.server_middleware do |chain|
chain.remove Sidekiq::Middleware::Server::RetryJobs
end
end
class S3Worker
include Sidekiq::Worker
def perform
@storage = Fog::Storage.new(
provider: 'AWS',
aws_access_key_id: ENV['key'],
aws_secret_access_key: ENV['secret'],
path_style: true
)
1000.times do |i|
mime_type = 'text/html'
strio = StringIO.open('', 'w')
compressor = Zlib::GzipWriter.new(strio)
compressor.write('<html></html>')
compressor.close
params = {
"x-amz-acl" => "public-read",
"Content-Type" => mime_type,
"Content-Encoding" => 'gzip',
"Expires" => 60000
}
@storage.put_object(ENV['bucket'], "deadlock/#{i}.html", strio.string, params)
end
end
end
Raw
sidekiq.yml
---
:verbose: true
:concurrency: 5
:timeout: 5
:queues:
- default
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment