Skip to content

Instantly share code, notes, and snippets.

@vishwasnavadak
Last active July 16, 2019 17:19
Show Gist options
  • Save vishwasnavadak/bf461449edd7683b1535de60d7815e13 to your computer and use it in GitHub Desktop.
Save vishwasnavadak/bf461449edd7683b1535de60d7815e13 to your computer and use it in GitHub Desktop.
Scraping the web with Nodejs. Code for the blog post -> https://vishwas.tech/blog/2019/07/12/scraping-with-nodejs.html
const axios = require("axios");
const cheerio = require("cheerio");
const scrape = async url => {
const { data } = await axios.get(url).catch(err => console.log(err));
const $ = cheerio.load(data);
const result = $(".skill-col").text();
return result;
};
(async ()=> {
console.log(await scrape("https://vishwas.tech/"))
} )()
const Nightmare = require("nightmare");
const nightmare = Nightmare({ show: true });
nightmare
.goto("https://duckduckgo.com")
.type("#search_form_input_homepage", "how to scrape using nightmarejs")
.click("#search_button_homepage")
.wait("#r1-0 a.result__a")
.evaluate(() => document.querySelector("#r1-0 a.result__a").href)
.end()
.then(console.log)
.catch(error => {
console.error("Search failed:", error);
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment