A Python script to download all the tweets of a hashtag into a csv

twitter crawler.txt

import tweepy
import csv
import pandas as pd
####input your credentials here
consumer_key = ''
consumer_secret = ''
access_token = ''
access_token_secret = ''

auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth,wait_on_rate_limit=True)
#####United Airlines
# Open/Create a file to append data
csvFile = open('ua.csv', 'a')
#Use csv Writer
csvWriter = csv.writer(csvFile)

for tweet in tweepy.Cursor(api.search,q="#unitedAIRLINES",count=100,
                           lang="en",
                           since="2017-04-03").items():
    print (tweet.created_at, tweet.text)
    csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8')])

How to retrieve the name of the user who tweeted?

How can I get tweets containing only keywords I mentioned independent of position, like I tried stream.filter(track=["good","bank","@NYC"]), I used these keywords in the code but sometimes I am getting tweets containing either 1,2 or 3 of them together. I want to get tweets containing 3 of these keywords, independent of the position.

I am a novice here. The code didn't work for me first. However, it worked after I added one line to the end - csvFile.close()
Not sure if I am the only one who faced it or it is such a trivial problem that all others have fixed it already in their own codes.

This code works. Thank you!
How can I increase the character limit that is currently forced on the tweets extracted?

Thank you!

What is the max value to which I can set the count to?

Thanks! This is short and sweet. Worked for me using: Python 3.7.6

also, for those wondering how to get output into a Pandas DF, I did this (added two lists):

created_at = []
text = []
for tweet in tweepy.Cursor(api.search,q=["#covidnurse"],count=100,
                           lang="en",
                           since="2020-03-15").items():
    print (tweet.created_at, tweet.text)
    csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8')])
    created_at.append(tweet.created_at)
    text.append(tweet.text)

df =  pd.DataFrame(
    {'created_at': created_at,
     'tweet': text
    })
df #RT (Retweets) are duplicates since Twitter is an echo chamber; will need to get rid of duplicates

df_clean = df[~df.tweet.str.contains("RT")]
df_clean

Great work!!
I have a question- will it return any response if there is one only tweet which is dated Feb 19, 2015? For hashtag- LexusEnformRemote
I have read Tweepy only return tweets for the past 7 days.
One more thing, how to search for multiple hashtags in a single loop?

Answering my own question of getting tweets beyond the past 7 days. I used GetOldTweets3 library by @Jefferson-Henrique and it's working fine for me. We can get tweets by multiple user accounts in one go but search doesn't take two hashtags.
Link

@abhishek-negi to search for multiple hashtags you could perform a loop from a list.

Something like this:

list_tweets_all_hashtags = []
hashtags_list = #tag1, #tag2, #tagN

for hashtag in hashtags_list:
    list_by_tag = [for tweet in tweepy.Cursor(api.search, q=hashtag, result_type="recent").items(100)]
    list_tweets_all_hashtags.extend(list_by_tag)
    
print(list_tweets_all_hashtags)

I am getting the below error at this line: csvFile = open('askgretawhy.csv', 'a')

Traceback (most recent call last):
File "", line 1, in
PermissionError: [Errno 13] Permission denied: 'askgretawhy.csv'

Please help me understand where is the issue.

I am getting the below error at this line: csvFile = open('askgretawhy.csv', 'a')

Traceback (most recent call last):
File "", line 1, in
PermissionError: [Errno 13] Permission denied: 'askgretawhy.csv'

Please help me understand where is the issue.

It’s most likely you don’t have the appropriate permissions to write/overwrite file askgretawhy.csv or to write on the current folder where the program runs.

If the file already exists, I suggest to run chmod 777 askgretawhy.csv and try again. If not, try giving less strict permissions to the current folder, if you can.

I am getting the below error at this line: csvFile = open('askgretawhy.csv', 'a')
Traceback (most recent call last):
File "", line 1, in
PermissionError: [Errno 13] Permission denied: 'askgretawhy.csv'
Please help me understand where is the issue.

It’s most likely you don’t have the appropriate permissions to write/overwrite file askgretawhy.csv or to write on the current folder where the program runs.

If the file already exists, I suggest to run chmod 777 askgretawhy.csv and try again. If not, try giving less strict permissions to the current folder, if you can.

How do I change the permissions? I am working on basic Python editor and do not know where can I create the file for Python to read it.

File "C:/Users/hp/Documents/1.py", line 5
consumer_key ='249QYJU8UqoS364VqIOoWmpJQ'
^
IndentationError: unexpected indent

Getting this error please help
ASAP!

@riiyasharmaa you might need to unindent (tab to the left)

Many thanks for the crawler mate, I was looking for it and didn´t have time to spend on it.

cheers

I get This error from Python :
SyntaxError: (unicode error) 'unicodeescape' codec can't decode bytes in position 22-23: truncated \UXXXXXXXX escape
Any help please !!

Hello,
how we can add the geolocalisation like : France to this script and more filters in the search ???

AttributeError Traceback (most recent call last)
C:\Users\ADMINI~1\AppData\Local\Temp/ipykernel_11192/3895188609.py in
19 #Use csv Writer
20 csvWriter = csv.writer(csvFile)
---> 21 for tweet in tweepy.Cursor(api.search, q="#unitedAIRLINES",count=100, lang="en", since="2021-09-14").items():
22 print (tweet.created_at, tweet.text)
23 csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8')])

AttributeError: 'API' object has no attribute 'search'

I am using the latest version of Anaconda Navigator 2.10 and jupyter Notebook 6.4.4.

I have regenerated API tokens. How do should I fix this?

AttributeError Traceback (most recent call last) C:\Users\ADMINI~1\AppData\Local\Temp/ipykernel_11192/3895188609.py in 19 #Use csv Writer 20 csvWriter = csv.writer(csvFile) ---> 21 for tweet in tweepy.Cursor(api.search, q="#unitedAIRLINES",count=100, lang="en", since="2021-09-14").items(): 22 print (tweet.created_at, tweet.text) 23 csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8')])

AttributeError: 'API' object has no attribute 'search'

I am using the latest version of Anaconda Navigator 2.10 and jupyter Notebook 6.4.4.

I have regenerated API tokens. How do should I fix this?

I am getting the exact error reported above.

I had to upgrade my version of Anaconda and then installed tweepy version 4.1.0 and from there the following command is not working for me:

tweepy.Cursor(api.search, q=ht, since=since_date_str).items()

Any idea how to solve it? do I have to go back to a previous version of tweepy? if so, which version?