Create a gist now

Instantly share code, notes, and snippets.

Download all tweets from the twitter search API for a given search term (limited to their maximum of 1500)
#! /usr/bin/env ruby
require "fileutils"
search_term = ARGV[0]
if search_term
time = Time.now
directory_path = File.dirname(__FILE__) + "/tweets/" + search_term + "_" + time.to_i.to_s
FileUtils.mkdir_p(directory_path)
directory = Dir.new(directory_path)
(1..15).each do |i|
`curl "http://search.twitter.com/search.json?q=#{search_term}&rpp=100&page=#{i}&include_entities=true&result_type=mixed" > #{directory.path}/#{i}.json`
end
puts "Scraped to #{directory_path}"
else
puts "./scrape_tweets.rb <your search term in quotes>"
end
@jadedgnome

Can you make it download images too?

@chid
chid commented May 23, 2014

To anyone who finds this, note that this does not work anymore as Twitter has disabled their 1.0 API and now requires authentication for searches https://dev.twitter.com/docs/auth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment