public
Last active

Twitter search

  • Download Gist
twitter_search.R
R
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44
doInstall <- TRUE
toInstall <- c("twitteR", "lubridate")
if(doInstall){install.packages(toInstall, repos = "http://cran.us.r-project.org")}
lapply(toInstall, library, character.only = TRUE)
 
searchTerms <- c("New York", "Los Angeles", "Chicago", "Houston", "Philadelphia",
"Phoenix", "San Antonio", "San Diego", "Dallas", "San Jose",
"Jacksonville", "Indianapolis", "Austin", "San Francisco",
"Columbus", "Fort Worth", "Charlotte", "Detroit", "El Paso",
"Memphis")
 
names(searchTerms) <- searchTerms
 
searchResults <- lapply(searchTerms, function(tt){
print(tt)
searchTwitter(searchString = tt, n = 1000) # Get all Tweets with each search term
})
 
tweetFrames <- lapply(searchResults, twListToDF) # Convert to Tweet DFs.
tweetFrames <- lapply(tweetFrames, function(df){
df$timeStamp <- ymd_hms(as.character(df$created)) # lubridate the timestamps
return(df)
})
 
timeElapsed <- unlist(lapply(tweetFrames, function(df){
as.numeric(diff(range(df$timeStamp)), units = "secs")
}))
 
nTweets <- unlist(lapply(tweetFrames, function(df){
nrow(df) # Sometimes this is less than the n requested with searchTwitter()
}))
 
tweetsPerSec <- nTweets / timeElapsed
 
# Population counts, from: http://en.wikipedia.org/wiki/List_of_United_States_cities_by_population
populationCounts <- c(8175133, 3792621, 2695598, 2099451, 1526006, 1445632,
1327407, 1307402, 1197816, 945942, 821784, 820445, 790390,
805235, 787033, 741206, 731424, 719777, 649121, 646889)
 
comparisonFrame <- data.frame(TPS = tweetsPerSec)
comparisonFrame$Population <- populationCounts
 
with(comparisonFrame, plot(log(Population), TPS, type = "n"))
with(comparisonFrame, text(log(Population), TPS, rownames(comparisonFrame)))

Why should the code returns the error : Error in .self$twFromJSON(out) :
Error: Malformed response from server, was not JSON.

What does it means?
Thanks

I'm having the same problem.

Cheers

n = 1000 is just too big. reduce it. with 100 it's ok

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.