Skip to content

Instantly share code, notes, and snippets.

@sorah
Created May 9, 2013 16:54
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sorah/5548793 to your computer and use it in GitHub Desktop.
Save sorah/5548793 to your computer and use it in GitHub Desktop.
#!/usr/bin/env ruby
Dir.chdir(File.dirname(__FILE__))
require 'digest/sha2'
require 'uri'
require 'open-uri'
require 'json'
API_KEY = 'api_key'
TUMBLELOG = 'scrap.sorah.jp'
def get_photos(offset=0)
response = open("http://api.tumblr.com/v2/blog/#{TUMBLELOG}/posts/photo?api_key=#{API_KEY}&offset=#{offset}", 'r', &:read)
JSON.parse(response)
end
def generate_filename(post_url, img_url)
m = post_url.match(%r{/post/(\d+?)(\Z|/)})
post_id = m ? m[1] : ''
img_ext = File.extname(URI.parse(img_url).path)
"#{post_id}_#{Digest::SHA256.hexdigest(img_url)}#{img_ext}"
end
downloaded = true
offset = 0
while downloaded
downloaded = false
payload = get_photos(offset)
puts "============ Offset: #{offset}"
payload['response']['posts'].each do |item|
link = item['post_url']
puts "=> #{link}"
img_urls = item['photos'].map { |_| _['original_size']['url'] rescue nil }.compact
img_urls.each do |url|
filename = generate_filename(link, url)
if File.exist?(filename)
next
end
puts "* #{url} => #{filename}"
system "curl", "--progress-bar", "-o", filename, url
downloaded = true
end
end
offset += 20
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment