Skip to content

Instantly share code, notes, and snippets.

@matt-dray

matt-dray/rcrawler-purrr.R

Created August 1, 2018 14:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save matt-dray/9241c829b6040b97dc85aa926acd3173 to your computer and use it in GitHub Desktop.
Save matt-dray/9241c829b6040b97dc85aa926acd3173 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
rcrawler-purrr.R
library(dplyr)
library(Rcrawler)
library(purrr)
library(httr)
x <- c(
"https://rostrum.blog",
"https://therostrumblog.wordpress.com/"
)
crawl_vector <- function(x){
map(
x,
Rcrawler,
KeywordsFilter = c("dray", "adriana"), DIR = "test_html"
) %>%
setNames(x) %>%
map_df(bind_rows, .id="id")
}
crawl_vector(x)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment