-
-
Save shubhamcodez/0e51a5ee9a8d3115ddad96edaca7e1ac to your computer and use it in GitHub Desktop.
# Make sure you pip install snscrape pandas | |
import snscrape.modules.twitter as sntwitter | |
import pandas as pd | |
#Enter usersnames of the users whose tweets you need in the list | |
users = ['elonmusk'] | |
#Enter the number of tweets per user you'd like. Note: In case user has less than limit, all the tweets get downloaded. | |
limit = 10000 | |
for user in users: | |
until = "2022-12-15" #Enter the date until you want the tweet for | |
since = "2022-01-01" #Enter the date from the tweets start getting queried | |
query = f"(from:{user}) until:{until} since:{since}" | |
tweets = [] | |
for tweet in sntwitter.TwitterSearchScraper(query).get_items(): | |
#print(vars(tweet)) #gets all the attributes retrieved. | |
if len(tweets) == limit: | |
break | |
else: | |
contents = [tweet.user.displayname, tweet.user.username, tweet.url, tweet.date, tweet.content, tweet.retweetCount, tweet.likeCount] #Instruction: Add more attributes here | |
tweets.append(contents) | |
df = pd.DataFrame(tweets, columns= ["Username","User handle", "Tweet", "Date of posting" ,"Text","Retweet count","Like count"]) #Instruction: add a new name for column added | |
df.to_csv(user+'tweets.csv') |
Works great, thanks!
You're welcome.
I think my problem is installing re
ERROR: Could not find a version that satisfies the requirement re (from versions: none)
ERROR: No matching distribution found for re
I am new to python so bear with me.
I tried installing re2 but that did not work either.
I think my problem is installing
re
ERROR: Could not find a version that satisfies the requirement re (from versions: none) ERROR: No matching distribution found for re
I am new to python so bear with me. I tried installing re2 but that did not work either.
Oh, I see. No worries.
To build re2, you need to install Visual C++14.
Download this: https://visualstudio.microsoft.com/visual-cpp-build-tools/
Then installed, the selected packages in this image and also, Windows 10 SDK package:
Once you have installed it, restart the pc and try installing re2 again.
Let me know if it works.
Actually, it turned out my problem was entirely different. I was using py
to run the script. I used python
and it is working fine.
Some modifications I made on MacOS:
- (I don't write a lot of python) I had trouble with python trying to run as Python2. But, creating a venv fixed this issue for me -
python3 -m venv py3
to create, thensource py3/bin/activate
to activate (py3 is the environment name, you can call it anything) - It seems only
snscrape
andpandas
are actually used. So, installingnumpy
,re
, andos
aren't needed. Theimport
lines for those packages can be removed. - The new install command was
pip3 install snscrape pandas
Btw - thank you for this script. There's some giant repos on GitHub with dozens of dependencies. This is much lighter to run.
@Christopher-Hayes Thank you for making these changes. I am cursed with the habit of importing libraries. :p
script isn't working again
script isn't working again
I am sorry to inform you that twitter has updated its APIs, and none of the third-party scrapers will work.
script isn't working again
I am sorry to inform you that twitter has updated its APIs, and none of the third-party scrapers will work.
Yeah, snscraper still works though.
Works great, thanks!