Create a gist now

Instantly share code, notes, and snippets.

Embed
Gathering Tweets, geocoding users, and plotting them
doInstall <- TRUE
toInstall <- c("twitteR", "dismo", "maps", "ggplot2")
if(doInstall){install.packages(toInstall, repos = "http://cran.us.r-project.org")}
lapply(toInstall, library, character.only = TRUE)
searchTerm <- "#rstats"
searchResults <- searchTwitter(searchTerm, n = 1000) # Gather Tweets
tweetFrame <- twListToDF(searchResults) # Convert to a nice dF
userInfo <- lookupUsers(tweetFrame$screenName) # Batch lookup of user info
userFrame <- twListToDF(userInfo) # Convert to a nice dF
locatedUsers <- !is.na(userFrame$location) # Keep only users with location info
locations <- geocode(userFrame$location[locatedUsers]) # Use amazing API to guess
# approximate lat/lon from textual location data.
with(locations, plot(lon, lat))
worldMap <- map_data("world") # Easiest way to grab a world map shapefile
zp1 <- ggplot(worldMap)
zp1 <- zp1 + geom_path(aes(x = long, y = lat, group = group), # Draw map
colour = gray(2/3), lwd = 1/3)
zp1 <- zp1 + geom_point(data = locations, # Add points indicating users
aes(x = lon, y = lat),
colour = "RED", alpha = 1/2, size = 1)
zp1 <- zp1 + coord_equal() # Better projections are left for a future post
zp1 <- zp1 + theme_minimal() # Drop background annotations
print(zp1)
@ronin78

This comment has been minimized.

Show comment
Hide comment
@ronin78

ronin78 Dec 21, 2012

Great example, thanks! I am, however, getting an error on
>searchResults <- searchTwitter(searchTerm, n = 1000) Error in .self$twFromJSON(out) : Error: Malformed response from server, was not JSON

After Googling, this may be due to non-Latin characters in some of the tweets. Do you have any suggestions on how to modify the above example so that it is robust to this problem?

Thanks,
Matt

ronin78 commented Dec 21, 2012

Great example, thanks! I am, however, getting an error on
>searchResults <- searchTwitter(searchTerm, n = 1000) Error in .self$twFromJSON(out) : Error: Malformed response from server, was not JSON

After Googling, this may be due to non-Latin characters in some of the tweets. Do you have any suggestions on how to modify the above example so that it is robust to this problem?

Thanks,
Matt

@ghost

This comment has been minimized.

Show comment
Hide comment
@ghost

ghost Dec 21, 2012

I received the same error. Mine works as long as I run the package "RJSONIO".

ghost commented Dec 21, 2012

I received the same error. Mine works as long as I run the package "RJSONIO".

@ronin78

This comment has been minimized.

Show comment
Hide comment
@ronin78

ronin78 Dec 24, 2012

Thank you! I will try that.

ronin78 commented Dec 24, 2012

Thank you! I will try that.

@LittleOrangeC

This comment has been minimized.

Show comment
Hide comment
@LittleOrangeC

LittleOrangeC Dec 26, 2012

I had the same error. Installing RJSONIO did not seem to help.

I had the same error. Installing RJSONIO did not seem to help.

@rossmounce

This comment has been minimized.

Show comment
Hide comment
@rossmounce

rossmounce Jan 3, 2013

you need to install AND load RJSONIO.

So make sure you do

library(RJSONIO)

you need to install AND load RJSONIO.

So make sure you do

library(RJSONIO)

@ronin78

This comment has been minimized.

Show comment
Hide comment
@ronin78

ronin78 Jan 3, 2013

RJSONIO did not help me either.

ronin78 commented Jan 3, 2013

RJSONIO did not help me either.

@swetaswain-08

This comment has been minimized.

Show comment
Hide comment
@swetaswain-08

swetaswain-08 Oct 12, 2016

I am getting error on this
locations <- geocode(userFrame$location[locatedUsers])
failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

please help me out with this error.

I am getting error on this
locations <- geocode(userFrame$location[locatedUsers])
failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

please help me out with this error.

@srp140830

This comment has been minimized.

Show comment
Hide comment
@srp140830

srp140830 Oct 17, 2016

Hi,
Can someone guide me with the following error and how to resolve this error

failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

Hi,
Can someone guide me with the following error and how to resolve this error

failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

@biancaglez

This comment has been minimized.

Show comment
Hide comment
@biancaglez

biancaglez Dec 1, 2016

Hi! I am also having the same error as last two comments above:

failed to load HTTP resource
Error : 1: failed to load HTTP resource
Error: object 'doc' not found

I've loaded XML, httr, and Rcurl to try to amend this error. No luck thus far.
Any advice?

Hi! I am also having the same error as last two comments above:

failed to load HTTP resource
Error : 1: failed to load HTTP resource
Error: object 'doc' not found

I've loaded XML, httr, and Rcurl to try to amend this error. No luck thus far.
Any advice?

@vutla04

This comment has been minimized.

Show comment
Hide comment
@vutla04

vutla04 Dec 2, 2016

Hi,
facing same issue.

failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

vutla04 commented Dec 2, 2016

Hi,
facing same issue.

failed to load HTTP resource
Error : 1: failed to load HTTP resource

Error: object 'doc' not found

@MarkMellink

This comment has been minimized.

Show comment
Hide comment
@MarkMellink

MarkMellink Apr 5, 2017

Hi guys, for those with the problem of vutla04 and above, the filtering in line 13 seemed to be the problem for me.

I used the data.table library and changed this:
locatedUsers <- !is.na(userFrame$location) # Keep only users with location info
locations <- geocode(userFrame$location[locatedUsers]) # Use amazing API to guess approximate lat/lon from textual location data.

to this:
library(data.table)
locations <- geocode(dt.user.info$location[!dt.user.info$location %in% ""])

Problem is that blank locations are not seen as NA, maybe this changed recently.

Also note that the column names returned by the API for longitude and latitude, have changed. Previously it was lon and lat. It is now longitude and latitude.

Hi guys, for those with the problem of vutla04 and above, the filtering in line 13 seemed to be the problem for me.

I used the data.table library and changed this:
locatedUsers <- !is.na(userFrame$location) # Keep only users with location info
locations <- geocode(userFrame$location[locatedUsers]) # Use amazing API to guess approximate lat/lon from textual location data.

to this:
library(data.table)
locations <- geocode(dt.user.info$location[!dt.user.info$location %in% ""])

Problem is that blank locations are not seen as NA, maybe this changed recently.

Also note that the column names returned by the API for longitude and latitude, have changed. Previously it was lon and lat. It is now longitude and latitude.

@azkajavaid

This comment has been minimized.

Show comment
Hide comment
@azkajavaid

azkajavaid Apr 17, 2017

For those experiencing the "Malformed response from server, was not JSON" error, try searching for "rstats" rather than "#rstats." This worked for me: searchResults <- searchTwitter("rstats", n = 1000).

For those experiencing the "Malformed response from server, was not JSON" error, try searching for "rstats" rather than "#rstats." This worked for me: searchResults <- searchTwitter("rstats", n = 1000).

@salvatorst

This comment has been minimized.

Show comment
Hide comment
@salvatorst

salvatorst Jul 2, 2017

Hi, I also tried ALL suggestions above, but I still get this error:


locations <- geocode(userFrame$location[locatedUsers])
failed to load HTTP resource
Error : 1: failed to load HTTP resource


any solution? Thanks

Hi, I also tried ALL suggestions above, but I still get this error:


locations <- geocode(userFrame$location[locatedUsers])
failed to load HTTP resource
Error : 1: failed to load HTTP resource


any solution? Thanks

@SUshmitha93

This comment has been minimized.

Show comment
Hide comment
@SUshmitha93

SUshmitha93 Apr 4, 2018

Hi,
I tried all suggestions above, but I still get this error:

Error : 1: failed to load HTTP resource

Error in .geocode(xx$place, oneRecord = oneRecord, extent = extent, progress = progress) :
object 'doc' not found

Any Solutions ?? Thanks in Advance

Hi,
I tried all suggestions above, but I still get this error:

Error : 1: failed to load HTTP resource

Error in .geocode(xx$place, oneRecord = oneRecord, extent = extent, progress = progress) :
object 'doc' not found

Any Solutions ?? Thanks in Advance

@harsimratK

This comment has been minimized.

Show comment
Hide comment
@harsimratK

harsimratK Apr 19, 2018

For error: Error : 1: failed to load HTTP resource
Use
locations <- geocode(locatedUsers)

instead of locations <- geocode(userFrame$location[locatedUsers])

For error: Error : 1: failed to load HTTP resource
Use
locations <- geocode(locatedUsers)

instead of locations <- geocode(userFrame$location[locatedUsers])

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment