import tweepy | |
import csv | |
import pandas as pd | |
####input your credentials here | |
consumer_key = '' | |
consumer_secret = '' | |
access_token = '' | |
access_token_secret = '' | |
auth = tweepy.OAuthHandler(consumer_key, consumer_secret) | |
auth.set_access_token(access_token, access_token_secret) | |
api = tweepy.API(auth,wait_on_rate_limit=True) | |
#####United Airlines | |
# Open/Create a file to append data | |
csvFile = open('ua.csv', 'a') | |
#Use csv Writer | |
csvWriter = csv.writer(csvFile) | |
for tweet in tweepy.Cursor(api.search,q="#unitedAIRLINES",count=100, | |
lang="en", | |
since="2017-04-03").items(): | |
print (tweet.created_at, tweet.text) | |
csvWriter.writerow([tweet.created_at, tweet.text.encode('utf-8')]) |
This comment has been minimized.
This comment has been minimized.
When I run this script on Python 27 (on a windows 10 machine) nothing happens and no error is returned... can you please help? |
This comment has been minimized.
This comment has been minimized.
@vitospinelli When you see |
This comment has been minimized.
This comment has been minimized.
I tried using the same code on 2018-03-21 and was thinking to fetch tweet as old as 2018-02-01 but it was unable to return those many tweets, any idea why? |
This comment has been minimized.
This comment has been minimized.
@sushonvan1 the twitter API only lets you go back approximately 2 weeks |
This comment has been minimized.
This comment has been minimized.
Thanku very much |
This comment has been minimized.
This comment has been minimized.
i want to fetch tweet as old as 2018-01-01, can you help me please? |
This comment has been minimized.
This comment has been minimized.
Hi, I want to extract the hashtags from the tweets and store it into a file. is it possible? |
This comment has been minimized.
This comment has been minimized.
Hi, i want to save the tweets that i obtain into an array, is it possible?! thanks |
This comment has been minimized.
This comment has been minimized.
Running this script w/ Python 3.6, it's working just fine, outputting the data and creating the CSV file, but the CSV file appears empty when I open it. Any ideas? |
This comment has been minimized.
This comment has been minimized.
Thanks, the script runs fine on Python 3.5.2. I just would add a .encode('utf-8') in the print command on line 22. Otherwise I got an error when printing tweets to the console. |
This comment has been minimized.
This comment has been minimized.
How would I go about adding in location(e.g. geo_id or coordinates) and user_id? I've been going through the Tweepy documentation and Twitter API documentation, but can't find any information to add arguments like tweet.text and tweet.created_at. Update: After some more digging, I managed to find this output of the json file to find which arguments can be called for information: https://gist.github.com/dev-techmoe/ef676cdd03ac47ac503e856282077bf2 So, I learned that I can call geo, place, and coordinates (tweet.geo, tweet.place, tweet.coordinates), but it doesn't appear to do well for historical data. I maybe pulled 3 out of several thousand so far :/ But it's a handy reference for things like tweet.user.id or tweet.user.screen_name! I'm still looking for a way to determine if the tweet is a retweet (tweets I'd like to remove in my analysis), but besides the tweet.text beginning with "b'RT @" or if it is an advertisement (e.g. 'Buy 3 for 2 promotion' kinda thing). If anyone has any advice on those, I'd be greatly appreciative! |
This comment has been minimized.
This comment has been minimized.
@carlvlewis I had the same problem initially. I changed line 15 from append ('a') to write ('w') and it works just fine. |
This comment has been minimized.
This comment has been minimized.
it working fine...but no data is stored in the file.... |
This comment has been minimized.
This comment has been minimized.
having the same issue, nothing is being stored in csv, csnt seem to find the issue |
This comment has been minimized.
This comment has been minimized.
Thank you so much! |
This comment has been minimized.
This comment has been minimized.
Hello. I'm geting this error: File "C:\Users\Gabriel\Desktop\tweet\crawling.py", line 22, in Can someone help me? |
This comment has been minimized.
This comment has been minimized.
hello, pls how do i get the number of times a user posted a tweet using a particular hashtag |
This comment has been minimized.
This comment has been minimized.
if I want to add another attribute for streaming results, where I can see the tutorial. thank you |
This comment has been minimized.
This comment has been minimized.
Thank you! Works fine! |
This comment has been minimized.
This comment has been minimized.
Im getting error ~\AppData\Local\Continuum\anaconda3\lib\site-packages\tweepy\cursor.py in next(self) ~\AppData\Local\Continuum\anaconda3\lib\site-packages\tweepy\cursor.py in next(self) ~\AppData\Local\Continuum\anaconda3\lib\site-packages\tweepy\cursor.py in next(self) ~\AppData\Local\Continuum\anaconda3\lib\site-packages\tweepy\binder.py in _call(*args, **kwargs) ~\AppData\Local\Continuum\anaconda3\lib\site-packages\tweepy\binder.py in execute(self) TweepError: Twitter error response: status code = 401 |
This comment has been minimized.
This comment has been minimized.
In line 23, instead of CsvWriter it should be csvFile. |
This comment has been minimized.
This comment has been minimized.
@Balajigentela you should run in cmd |
This comment has been minimized.
This comment has been minimized.
please help me, is giving me this error D:\anaconda3\lib\site-packages\tweepy\cursor.py in next(self) D:\anaconda3\lib\site-packages\tweepy\cursor.py in next(self) D:\anaconda3\lib\site-packages\tweepy\binder.py in execute(self) |
This comment has been minimized.
This comment has been minimized.
It works fine for me! Thanks! |
This comment has been minimized.
This comment has been minimized.
Regenerate your API Keys and Token. it will fine |
This comment has been minimized.
This comment has been minimized.
I am having trouble with this code as it is spitting out an error that I am not sure how to solve: |
This comment has been minimized.
This comment has been minimized.
what if i want to extract tweets from more number of hashtags? should i pass list of hashtags to 'q' attribute of tweepy.cursor? |
This comment has been minimized.
This comment has been minimized.
how to search for multiple hashtags? |
This comment has been minimized.
This comment has been minimized.
thank you thank you thank you! |
This comment has been minimized.
This comment has been minimized.
Please help me.. I am getting thefollowing error.. C:\Users\geethu\LangDetect\Scripts\python.exe "C:/Users/geethu/PycharmProjects/Landslip_langDetection_Final/twitter crawler.py" During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): During handling of the above exception, another exception occurred: Traceback (most recent call last): Process finished with exit code 1 |
This comment has been minimized.
This comment has been minimized.
i couldn't find "since" parameter in tweepy documents. would you mind to explain and give me a link to the corresponding document? |
This comment has been minimized.
This comment has been minimized.
ua.csv is an empty file ...can you please guide on how to get the data in ua.csv |
This comment has been minimized.
This comment has been minimized.
seems to down load tweets, but most appear to be chopped off so you don't get the full tweet. Not very useful. |
This comment has been minimized.
This comment has been minimized.
@ateeley , you may use my code which uses the extended twitter feature and you can get the full tweet. https://gist.github.com/sxshateri/540aead254bfa7810ee8bbb2d298363e |
This comment has been minimized.
This comment has been minimized.
thanks sxshateri, seems to work ok with python3 in virtuelenv. |
This comment has been minimized.
This comment has been minimized.
Hi, Thanks for this code. I am a beginner, learning python, so please bear with my list of questions. |
This comment has been minimized.
This comment has been minimized.
How do i do that |
This comment has been minimized.
This comment has been minimized.
yes it is possible |
This comment has been minimized.
This comment has been minimized.
Thank you, this is really valuable, i was going thru another path but this is shows that it can be done in a simple way. |
This comment has been minimized.
This comment has been minimized.
Great work!! One more thing, how to search for multiple hashtags in a single loop? |
This comment has been minimized.
This comment has been minimized.
Answering my own question of getting tweets beyond the past 7 days. I used GetOldTweets3 library by @Jefferson-Henrique and it's working fine for me. We can get tweets by multiple user accounts in one go but search doesn't take two hashtags. |
This comment has been minimized.
This comment has been minimized.
hey i am also getting the same error did u sove that |
This comment has been minimized.
This comment has been minimized.
Hello, It's my first time with python and this library and I have a strange error. In my query, i don't have all the tweets, only the 25-30 last tweets. But if I do the same search in Twitter, I have a lot of tweets in the results. Anyone knows why? |
This comment has been minimized.
This comment has been minimized.
I am also facing same issues. Can you tell me how you fixed it? |
This comment has been minimized.
This comment has been minimized.
I am also facing same issues. Can you tell me how you fixed it? |
This comment has been minimized.
This comment has been minimized.
Hi! Is it possible to make this "live" such as everytime a tweet is posted with a #cat it fetches it? Or that the code just looks a the last minute every other minute? I hope this is clear? |
This comment has been minimized.
This comment has been minimized.
Can anyone help me find a way to obtain these data for longer period of time? such as using gotoldtweets3? Thank you very much. |
This comment has been minimized.
This comment has been minimized.
just signed up to github to say thank you |
This comment has been minimized.
This comment has been minimized.
Hello Thank You for the above code, I wanted to download data by hashtag and the location. For example - tweets with "#coronavirus" and the locations of the tweets "India" and the tweets time limit from "2020-04-01" till "2020-04-31", can anyone please help me for that, It is really important for my research. |
This comment has been minimized.
This comment has been minimized.
How to retrieve the name of the user who tweeted? |
This comment has been minimized.
This comment has been minimized.
How can I get tweets containing only keywords I mentioned independent of position, like I tried |
This comment has been minimized.
This comment has been minimized.
I am a novice here. The code didn't work for me first. However, it worked after I added one line to the end - csvFile.close() |
This comment has been minimized.
This comment has been minimized.
This code works. Thank you! Thank you! |
This comment has been minimized.
This comment has been minimized.
What is the max value to which I can set the count to? |
This comment has been minimized.
This comment has been minimized.
Thanks! This is short and sweet. Worked for me using: Python 3.7.6 also, for those wondering how to get output into a Pandas DF, I did this (added two lists):
|
This comment has been minimized.
This comment has been minimized.
@abhishek-negi to search for multiple hashtags you could perform a loop from a list. Something like this:
|
This comment has been minimized.
This comment has been minimized.
I am getting the below error at this line: csvFile = open('askgretawhy.csv', 'a') Traceback (most recent call last): Please help me understand where is the issue. |
This comment has been minimized.
This comment has been minimized.
It’s most likely you don’t have the appropriate permissions to write/overwrite file askgretawhy.csv or to write on the current folder where the program runs. If the file already exists, I suggest to run |
This comment has been minimized.
This comment has been minimized.
How do I change the permissions? I am working on basic Python editor and do not know where can I create the file for Python to read it. |
This comment has been minimized.
thank you!