Skip to content

Instantly share code, notes, and snippets.

@a-chernykh
Created June 28, 2014 10:11
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save a-chernykh/5a2b43e5c5cc583e8a69 to your computer and use it in GitHub Desktop.
Save a-chernykh/5a2b43e5c5cc583e8a69 to your computer and use it in GitHub Desktop.
require 'net/http'
def download_net_http(urls, thread_count)
queue = Queue.new
urls.map { |url| queue << url }
threads = thread_count.times.map do
Thread.new do
Net::HTTP.start('our-s3-bucket.s3.amazonaws.com', 80) do |http|
while !queue.empty? && url = queue.pop
uri = URI(url)
request = Net::HTTP::Get.new(uri)
response = http.request request
write_file url, response.body
end
end
end
end
threads.each(&:join)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment