public
Last active

Download all tweets from the twitter search API for a given search term (limited to their maximum of 1500)

  • Download Gist
scrape_tweets.rb
Ruby
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
#! /usr/bin/env ruby
require "fileutils"
search_term = ARGV[0]
if search_term
time = Time.now
directory_path = File.dirname(__FILE__) + "/tweets/" + search_term + "_" + time.to_i.to_s
FileUtils.mkdir_p(directory_path)
directory = Dir.new(directory_path)
(1..15).each do |i|
`curl "http://search.twitter.com/search.json?q=#{search_term}&rpp=100&page=#{i}&include_entities=true&result_type=mixed" > #{directory.path}/#{i}.json`
end
puts "Scraped to #{directory_path}"
else
puts "./scrape_tweets.rb <your search term in quotes>"
end

Can you make it download images too?

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.