Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Pull down a few public domain books
require 'net/http'
require 'uri'
(10000..10010).each do |i|
path = i.to_s.chars.to_a[0...-1].inject(""){|memo, value|"#{memo}/#{value}"}
path = "/dirs" + path + "/#{i}/#{i}.txt"
puts "Get: http://www.gutenberg.org" + path
Net::HTTP::Proxy("yourproxyserver", 8080).start("www.gutenberg.org", 80) do |http|
http.request_get(path) do |response|
File.open("#{i}.txt", 'w') do |file|
response.read_body { |body_string| file.write body_string }
end if response.is_a? Net::HTTPSuccess
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.