Skip to content

Instantly share code, notes, and snippets.

@evanwinter
Last active July 29, 2020 20:13
Show Gist options
  • Save evanwinter/f2c6c42996cf0dd8f1d45661ac949d6a to your computer and use it in GitHub Desktop.
Save evanwinter/f2c6c42996cf0dd8f1d45661ac949d6a to your computer and use it in GitHub Desktop.
Bare minimum script for scraping a website with Node.js
const fetch = require("node-fetch")
const { JSDOM } = require("jsdom")
const fetchHtml = async (url) => {
const response = await fetch(url)
const text = await response.text()
const dom = new JSDOM(text)
return dom.window.document
}
const extractElements = (document, selector) => {
const elements = Array.from(document.querySelectorAll(selector))
return elements.map((element) => element.textContent)
}
const scrape = async (url, selector) => {
const document = await fetchHtml(url)
return extractElements(document, selector)
}
scrape("https://evanwinter.dev", ".nav li").then((data) => console.log(data))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment