Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Gathering Tweets, geocoding users, and plotting them
doInstall <- TRUE
toInstall <- c("twitteR", "dismo", "maps", "ggplot2")
if(doInstall){install.packages(toInstall, repos = "http://cran.us.r-project.org")}
lapply(toInstall, library, character.only = TRUE)
searchTerm <- "#rstats"
searchResults <- searchTwitter(searchTerm, n = 1000) # Gather Tweets
tweetFrame <- twListToDF(searchResults) # Convert to a nice dF
userInfo <- lookupUsers(tweetFrame$screenName) # Batch lookup of user info
userFrame <- twListToDF(userInfo) # Convert to a nice dF
locatedUsers <- !is.na(userFrame$location) # Keep only users with location info
locations <- geocode(userFrame$location[locatedUsers]) # Use amazing API to guess
# approximate lat/lon from textual location data.
with(locations, plot(lon, lat))
worldMap <- map_data("world") # Easiest way to grab a world map shapefile
zp1 <- ggplot(worldMap)
zp1 <- zp1 + geom_path(aes(x = long, y = lat, group = group), # Draw map
colour = gray(2/3), lwd = 1/3)
zp1 <- zp1 + geom_point(data = locations, # Add points indicating users
aes(x = lon, y = lat),
colour = "RED", alpha = 1/2, size = 1)
zp1 <- zp1 + coord_equal() # Better projections are left for a future post
zp1 <- zp1 + theme_minimal() # Drop background annotations
print(zp1)

ronin78 commented Dec 21, 2012

Great example, thanks! I am, however, getting an error on
>searchResults <- searchTwitter(searchTerm, n = 1000) Error in .self$twFromJSON(out) : Error: Malformed response from server, was not JSON

After Googling, this may be due to non-Latin characters in some of the tweets. Do you have any suggestions on how to modify the above example so that it is robust to this problem?

Thanks,
Matt

@ghost

ghost commented Dec 21, 2012

I received the same error. Mine works as long as I run the package "RJSONIO".

ronin78 commented Dec 24, 2012

Thank you! I will try that.

I had the same error. Installing RJSONIO did not seem to help.

you need to install AND load RJSONIO.

So make sure you do

library(RJSONIO)

ronin78 commented Jan 3, 2013

RJSONIO did not help me either.

I am getting error on this
locations <- geocode(userFrame$location[locatedUsers])
failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

please help me out with this error.

Hi,
Can someone guide me with the following error and how to resolve this error

failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

Hi! I am also having the same error as last two comments above:

failed to load HTTP resource
Error : 1: failed to load HTTP resource
Error: object 'doc' not found

I've loaded XML, httr, and Rcurl to try to amend this error. No luck thus far.
Any advice?

vutla04 commented Dec 2, 2016

Hi,
facing same issue.

failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

Hi guys, for those with the problem of vutla04 and above, the filtering in line 13 seemed to be the problem for me.

I used the data.table library and changed this:
locatedUsers <- !is.na(userFrame$location) # Keep only users with location info
locations <- geocode(userFrame$location[locatedUsers]) # Use amazing API to guess approximate lat/lon from textual location data.

to this:
library(data.table)
locations <- geocode(dt.user.info$location[!dt.user.info$location %in% ""])

Problem is that blank locations are not seen as NA, maybe this changed recently.

Also note that the column names returned by the API for longitude and latitude, have changed. Previously it was lon and lat. It is now longitude and latitude.

ajav17 commented Apr 17, 2017

For those experiencing the "Malformed response from server, was not JSON" error, try searching for "rstats" rather than "#rstats." This worked for me: searchResults <- searchTwitter("rstats", n = 1000).

Hi, I also tried ALL suggestions above, but I still get this error:


locations <- geocode(userFrame$location[locatedUsers])
failed to load HTTP resource
Error : 1: failed to load HTTP resource


any solution? Thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment