Skip to content

Instantly share code, notes, and snippets.

@jpoechill
Created August 16, 2019 20:52
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save jpoechill/07c0f4c42a3d9e2c8927abfac68c6a40 to your computer and use it in GitHub Desktop.
Save jpoechill/07c0f4c42a3d9e2c8927abfac68c6a40 to your computer and use it in GitHub Desktop.
A simple client-side web scraper setup.
import axios from 'axios'
// a proxy server is not always needed for all sites
// though in some cases, sites will restrict access to those that are not same-origin
// in such case, proxy servers can be used to bypass, as seen here using https://cors-anywhere.herokuapp.com/.
axios.get('https://cors-anywhere.herokuapp.com/https://designboom.com', { crossdomain: true }).then((response) => {
// returns raw html which can be used subsequently as DOM nodes
console.log(response.data)
})
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment