Skip to content

Instantly share code, notes, and snippets.

@boyd
Created September 9, 2012 00:31
Show Gist options
  • Save boyd/3681601 to your computer and use it in GitHub Desktop.
Save boyd/3681601 to your computer and use it in GitHub Desktop.
Simple web request file caching system (similar to wget)
require 'FileUtils'
require 'net/http'
require 'uri'
class WebCache
def initialize
@download_cache_root = "./downloads"
end
def url_to_path(url)
uri = URI(url)
if uri.query
query = uri.query
else
query = "index.html"
end
return File.join(@download_cache_root, uri.host, uri.path, query)
end
def get(url)
file_path = url_to_path(url)
if not File.exists?file_path
# create directory if needed
dir_path = File.dirname(file_path)
if not File.exists?dir_path
FileUtils.mkdir_p(dir_path)
end
# download file
File.open(file_path, "w") do |data|
data << Net::HTTP.get_response(URI.parse(url)).body
end
end
return File.new(file_path).read
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment