Skip to content

Instantly share code, notes, and snippets.

@ryanmjacobs
Last active August 29, 2015 14:14
Show Gist options
  • Save ryanmjacobs/50a7944e929c1329ad44 to your computer and use it in GitHub Desktop.
Save ryanmjacobs/50a7944e929c1329ad44 to your computer and use it in GitHub Desktop.
Download all xkcd comics.
$ git clone https://gist.github.com/ryanmjacobs/50a7944e929c1329ad44 xkcd
$ cd xkcd
$ gem install colorize
$ chmod +x xkcd.rb
$ ./xkcd.rb

Some extra info if you're curious:

  • git clone ... xkcd grabs these files and puts them in a directory called xkcd.
  • cd xkcd goes into that directory.
  • gem install ... installs a Ruby gem. Ruby gems are small libraries and you can find one for pretty much anything.
  • chmod +x xkcd.rb makes the file xkcd.rb executable. Notice that the file turns green after you do it.
  • ./xkcd.rb executes the file. Pretty simple.
#!/usr/bin/env ruby
# dl all xkcd
require "json"
require "colorize"
require "open-uri"
Dir.mkdir "xkcd" if not Dir.exists?("xkcd")
$all_json = []
current_num = JSON.parse(open("http://xkcd.com/info.0.json").read)["num"].to_i
def quit!
File.open("xkcd/info.json", "w") { |f| f.write(JSON.pretty_generate($all_json)) }
end
def error(string)
puts "error: #{string}".red
File.open("xkcd.log", "a") { |f| f.write("#{Time.now.strftime("%F %T")} ERROR: #{string}\n") }
end
trap "SIGINT" do
quit!
exit 1
end
(1..current_num).each do |i|
begin
json = JSON.parse(open("http://xkcd.com/#{i}/info.0.json").read)
str_num = i.to_s.rjust(3, "0")
fname = "##{str_num} - #{json["safe_title"]}.png"
# make filename safe
fname.gsub!("/", "\u2215") # unicode slash
puts fname
$all_json << json
# numbers mismatch!
if i != json["num"].to_i then
abort "error: numbers mismatch! #{i} != #{json["num"].to_i}"
end
# skip if already downloaded
next if File.exists?("xkcd/#{fname}")
# write new image
File.open("xkcd/#{fname}", "wb") do |local|
open("#{json["img"]}", "rb") do |remote|
local.write(remote.read)
end
end
rescue Errno::ECONNRESET
error "##{str_num} - connection reset. skipping"
rescue OpenURI::HTTPError
error "##{str_num} - 404 not found. skipping"
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment