Skip to content

Instantly share code, notes, and snippets.

@Overbryd
Created January 28, 2012 19:33
Show Gist options
  • Save Overbryd/1695523 to your computer and use it in GitHub Desktop.
Save Overbryd/1695523 to your computer and use it in GitHub Desktop.
A very fast spaghetti-script written in Ruby that downloads files
#!/usr/bin/env ruby
# download (Works for me™)
#
# A simple & fast download script that utilizes em-http-request as http client
# Usage: download <url>
#
# The filename is determined either by the GET path or by the Content-Disposition response header if given.
# The script will check if there is an existing file and try to resume the download if possible.
#
# Sometimes writing Spaghetti code can be a lot of fun. Don't use it, I do not want to maintain this script.
require 'rubygems'
require 'eventmachine'
require 'em-http'
unless url = ARGV.first
puts "Usage: download <url>"
puts "Destination directory is hardcoded to ~/Downloads"
exit 1
end
filename = File.basename(url)
target = nil
content_length = received_bytes = 0
file = nil
resume_with_range = false
at_exit { file.close if file.kind_of?(File) && !file.closed? }
download = Proc.new do
EM.run do
redirects = 0
request_headers = { 'Accept-Encoding' => 'deflate, gzip' }
request_headers.merge!({ 'Range' => "bytes=#{received_bytes}-" }) if resume_with_range
http = EM::HttpRequest.new(url).get(:head => request_headers, :redirects => 10)
http.headers do
if http.response_header['LOCATION']
puts "Hop ##{redirects+=1} to #{http.response_header['LOCATION']}"
else
filename = (http.response_header['CONTENT_DISPOSITION'] =~ /filename="(.*)"/ && $1) || filename
target = File.expand_path(File.join('~/Downloads', filename))
unless resume_with_range
unfinished = File.exist?(target) && File.size(target) < content_length
file = File.exist?(target) ? File.open(target, 'r+') : File.open(target, 'w')
content_length = http.response_header['CONTENT_LENGTH'].to_i
if unfinished && http.response_header['ACCEPT_RANGES']
received_bytes = File.size(target)
resume_with_range = true
file.close
EM.stop
elsif unfinished
puts "Restarting download (Unfortunately the server you download from does not support resuming the transfer)"
file.rewind
elsif File.exist?(target) && File.size(target) == content_length
puts "Download already finished"
file.close
EM.stop
end
else
if http.response_header['CONTENT_RANGE'] =~ %r{bytes (\d+)-\d+/(\d+)}
filepos = $1.to_i
content_length = $2.to_i
else
filepos = 0
end
if filepos == received_bytes
puts "Resuming the transfer"
file.reopen(target, 'r+')
file.seek(filepos)
else
puts "Restarting download (Unfortunately the server you download from does not support resuming the transfer)"
received_bytes = 0
file.reopen(target, 'w')
end
end
file.sync = true unless file.closed?
end
end
http.stream do |chunk|
unless file.closed?
received_bytes += file.write(chunk)
print "\r#{'%0.2f' % (received_bytes / 1024.0 / 1024.0)}/#{'%0.2f' % (content_length / 1024.0 / 1024.0)} MB"
end
end
http.callback { EM.stop }
http.errback { EM.stop }
end
end
download.call
download.call if resume_with_range
puts target
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment