Skip to content

Instantly share code, notes, and snippets.

@erichannell
Last active August 12, 2016 20:54
Show Gist options
  • Save erichannell/434689526acb85f52c36 to your computer and use it in GitHub Desktop.
Save erichannell/434689526acb85f52c36 to your computer and use it in GitHub Desktop.
Grabbing things from Twitter as they fly by.
from TwitterSearch import *
import csv
def get_tweets(query, max = 2000):
# takes a search term (query) and a max number of tweets to find
# gets content from twitter and writes it to a csv bearing the name of your query
i = 0
search = query
with open(search+'.csv', 'wb') as outf:
writer = csv.writer(outf)
writer.writerow(['user','time','tweet','latitude','longitude'])
try:
tso = TwitterSearchOrder()
tso.set_keywords([search])
tso.set_language('en') # English tweets only
ts = TwitterSearch(
consumer_key = 'YOUR CONSUMER KEY',
consumer_secret = 'YOUR CONSUMER SECRET',
access_token = 'YOUR ACCESS TOKEN',
access_token_secret = 'YOUR ACCESS TOKEN SECRET'
)
for tweet in ts.search_tweets_iterable(tso):
lat = None
long = None
time = tweet['created_at'] # UTC time when Tweet was created.
user = tweet['user']['screen_name']
tweet_text = tweet['text'].strip().encode('ascii', 'ignore')
tweet_text = ''.join(tweet_text.splitlines())
print i,time,
if tweet['geo'] != None and tweet['geo']['coordinates'][0] != 0.0: # avoiding bad values
lat = tweet['geo']['coordinates'][0]
long = tweet['geo']['coordinates'][1]
print('@%s: %s' % (user, tweet_text)), lat, long
else:
print('@%s: %s' % (user, tweet_text))
writer.writerow([user, time, tweet_text, lat, long])
i += 1
if i > max:
return()
except TwitterSearchException as e: # take care of all those ugly errors if there are some
print(e)
query = raw_input ("Search for: ")
max_tweets = 2000
get_tweets(query, max_tweets)
@erichannell
Copy link
Author

If you get an error looking like: Error 401: ('Unauthorized: Authentication credentials ', ' were missing or incorrect') then your keys are wrong (see lines 20-23). Make sure to go back and get the correct ones from dev.twitter.com

@fdaudens
Copy link

Hi there,

Thank you very much for the tutorial! One edit: to get special caracters right, you have to modify a few lines:

  user = tweet['user']['screen_name']
            tweet_text = tweet['text'].strip()
            tweet_text = u''.join(tweet_text.splitlines())
            print i, time,
            if tweet['geo'] and tweet['geo']['coordinates'][0]: 
                lat, long = tweet['geo']['coordinates'][:2]
                print u'@%s: %s' % (user, tweet_text), lat, long
            else:
                print u'@%s: %s' % (user, tweet_text)

            writer.writerow([user.encode('utf8'), time.encode('utf8'), 
                             tweet_text.encode('utf8'), lat, long])

(and add # -- coding: utf-8 -- at the beginning of your script).

Thanks to Martijn Pieters who helped me a lot! http://stackoverflow.com/questions/29036944/encoding-error-with-twittersearch-in-python

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment