Skip to content

Instantly share code, notes, and snippets.

@stefl
Created March 25, 2012 14:47
Show Gist options
  • Save stefl/2196581 to your computer and use it in GitHub Desktop.
Save stefl/2196581 to your computer and use it in GitHub Desktop.
Download all tweets from the twitter search API for a given search term (limited to their maximum of 1500)
#! /usr/bin/env ruby
require "fileutils"
search_term = ARGV[0]
if search_term
time = Time.now
directory_path = File.dirname(__FILE__) + "/tweets/" + search_term + "_" + time.to_i.to_s
FileUtils.mkdir_p(directory_path)
directory = Dir.new(directory_path)
(1..15).each do |i|
`curl "http://search.twitter.com/search.json?q=#{search_term}&rpp=100&page=#{i}&include_entities=true&result_type=mixed" > #{directory.path}/#{i}.json`
end
puts "Scraped to #{directory_path}"
else
puts "./scrape_tweets.rb <your search term in quotes>"
end
@jadedgnome
Copy link

Can you make it download images too?

@chid
Copy link

chid commented May 23, 2014

To anyone who finds this, note that this does not work anymore as Twitter has disabled their 1.0 API and now requires authentication for searches https://dev.twitter.com/docs/auth

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment