Skip to content

Instantly share code, notes, and snippets.

@sxshateri
Last active July 27, 2022 07:55
Show Gist options
  • Star 9 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save sxshateri/540aead254bfa7810ee8bbb2d298363e to your computer and use it in GitHub Desktop.
Save sxshateri/540aead254bfa7810ee8bbb2d298363e to your computer and use it in GitHub Desktop.
This is a python script to download all the tweets of a hashtag into a csv file.
import tweepy
import csv
import pandas as pd
import sys
# API credentials here
consumer_key = 'INSERT CONSUMER KEY HERE'
consumer_secret = 'INSERT CONSUMER SECRET HERE'
access_token = 'INSERT ACCESS TOKEN HERE'
access_token_secret = 'INSERT ACCESS TOKEN SECRET HERE'
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_token_secret)
api = tweepy.API(auth,wait_on_rate_limit=True,wait_on_rate_limit_notify=True)
# Search word/hashtag value
HashValue = ""
# search start date value. the search will start from this date to the current date.
StartDate = ""
# getting the search word/hashtag and date range from user
HashValue = input("Enter the hashtag you want the tweets to be downloaded for: ")
StartDate = input("Enter the start date in this format yyyy-mm-dd: ")
# Open/Create a file to append data
csvFile = open(HashValue+'.csv', 'a')
#Use csv Writer
csvWriter = csv.writer(csvFile)
for tweet in tweepy.Cursor(api.search,q=HashValue,count=20,lang="en",since=StartDate, tweet_mode='extended').items():
print (tweet.created_at, tweet.full_text)
csvWriter.writerow([tweet.created_at, tweet.full_text.encode('utf-8')])
print ("Scraping finished and saved to "+HashValue+".csv")
#sys.exit()
@sxshateri
Copy link
Author

This script is written in python 3. Please make sure you have installed Tweepy and Pandas before running the script.

@meylanwongkar
Copy link

how can i save the tweets with header using pandas?

@sxshateri
Copy link
Author

how can i save the tweets with header using pandas?

what exactly do you mean by "header"? are you referring to the header row of the csv file?

@sameermahajan101
Copy link

how to filter unwanted characters b'RT ,&amp and many more in the csv file?

@smycek
Copy link

smycek commented Sep 29, 2019

@sxshateri can i use this script to download tweets whom matches two or more words/hashtags?

@smycek
Copy link

smycek commented Oct 11, 2019

@sxshateri can i use this script to download tweets whom matches two or more words/hashtags?

In the meantime, I've been able to answer this myself: Any operator who follows the guidelines for Twitter search can be used as input.could answer this by my self: any operators following the twitter search guidlines can be used as input.

@sxshateri
Copy link
Author

@sxshateri can i use this script to download tweets whom matches two or more words/hashtags?

In the meantime, I've been able to answer this myself: Any operator who follows the guidelines for Twitter search can be used as input.could answer this by my self: any operators following the twitter search guidlines can be used as input.

yes. as you mentioned any operator within the twitter search guideline can be used in this script.

@this-is-shashank
Copy link

I am unable to download tweets older than 2 weeks. Can anyone help here ?

@anuraglahon16
Copy link

I am unable to download tweets older than 2 weeks. Can anyone help here ?

R u able to do it?

@Psion05
Copy link

Psion05 commented Jul 31, 2020

I am unable to download tweets older than 2 weeks. Can anyone help here ?

R u able to do it?

It's because Twitter Standard(Free) API will only search back till 14 days, from time of running script. If you want more, you'll have to get premium API access

@vedantkalan777
Copy link

Hello Thank You for the above code, I wanted to download data by hashtag and the location. For example - tweets with "#coronavirus" and the locations of the tweets "India" and the tweets time limit from "2020-04-01" till "2020-04-31", can anyone please help me for that, It is really important for my research.

@utkarshumang99
Copy link

How can I extract tweets with two #hashtags in common???

@longyi-yang
Copy link

Can I set multiple hashtags?

@dhilataha
Copy link

where is the file of tweets i downloaded?

@dhilataha
Copy link

@sxshateri can i use this script to download tweets whom matches two or more words/hashtags?

In the meantime, I've been able to answer this myself: Any operator who follows the guidelines for Twitter search can be used as input.could answer this by my self: any operators following the twitter search guidlines can be used as input.

would you tell me the script to do it?

@sxshateri
Copy link
Author

sxshateri commented Dec 1, 2020

Hello Thank You for the above code, I wanted to download data by hashtag and the location. For example - tweets with "#coronavirus" and the locations of the tweets "India" and the tweets time limit from "2020-04-01" till "2020-04-31", can anyone please help me for that, It is really important for my research.

@vedantkalan777 You won't be able to get the tweets older than 14 days with Twitter's free API.

@sxshateri
Copy link
Author

where is the file of tweets i downloaded?

@dhilataha where ever your script file is will be a .csv file with the same name as the hashtag you entered.

@dhilataha
Copy link

dhilataha commented Dec 1, 2020 via email

@ifeomaaaa
Copy link

ifeomaaaa commented Jul 27, 2022

I runs the program in google colaboratory then i dont know where is my #hashtag.csv file. I search it all over my computer's document but i cant find it Heavenly Regard Fadhillah Putri Taha Pada tanggal Sel, 1 Des 2020 08.14, Shahin Shateri notifications@github.com menulis:

@.**** commented on this gist. ------------------------------ where is the file of tweets i downloaded? @dhilataha https://github.com/dhilataha where ever your script file is will be a .csv file with the same name as the hashtag you entered. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://gist.github.com/540aead254bfa7810ee8bbb2d298363e#gistcomment-3545483, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH2ZFBY44FON34QYWCRDTBDSSQYOZANCNFSM4HYAEFAA .

I have this same issue.. did you ever find it?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment