Skip to content

Instantly share code, notes, and snippets.

@newportandy
Created July 22, 2010 22:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save newportandy/486694 to your computer and use it in GitHub Desktop.
Save newportandy/486694 to your computer and use it in GitHub Desktop.
Pull down a few public domain books
require 'net/http'
require 'uri'
(10000..10010).each do |i|
path = i.to_s.chars.to_a[0...-1].inject(""){|memo, value|"#{memo}/#{value}"}
path = "/dirs" + path + "/#{i}/#{i}.txt"
puts "Get: http://www.gutenberg.org" + path
Net::HTTP::Proxy("yourproxyserver", 8080).start("www.gutenberg.org", 80) do |http|
http.request_get(path) do |response|
File.open("#{i}.txt", 'w') do |file|
response.read_body { |body_string| file.write body_string }
end if response.is_a? Net::HTTPSuccess
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment