Skip to content

Instantly share code, notes, and snippets.

@cmeiklejohn
Created July 21, 2010 16:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cmeiklejohn/484674 to your computer and use it in GitHub Desktop.
Save cmeiklejohn/484674 to your computer and use it in GitHub Desktop.
Threading problems... help?
#!/usr/bin/ruby
require 'openssl'
require 'base64'
require 'rubygems'
require 'aws'
# Open master s3 connection
master_s3 = Aws::S3.new(
'key',
'secret_key'
)
# Setup queue
q = Queue.new
# Select the bucket
bucket = master_s3.bucket('assets-bucket-name')
# Iterate over the keys
bucket.keys.each do |key|
while q.size >= 60
puts "sleeping because queue size is #{q.size}"
sleep 1
end
q << Thread.new(key) do |key|
puts "Working on #{key}"
Aws::S3::Grantee.new(key, 'user_id_of_cloudfront_user', 'READ', :apply)
q.pop
end
end
# It freezes here... but only after multiple thousands of threads...
until q.empty?
sleep 1
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment