Skip to content

Instantly share code, notes, and snippets.

@saasindustries
Created February 1, 2021 16:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save saasindustries/203a6cf476e18bc803b84c5a4759347a to your computer and use it in GitHub Desktop.
Save saasindustries/203a6cf476e18bc803b84c5a4759347a to your computer and use it in GitHub Desktop.
const puppeteer = require('puppeteer');
(async function scrape() {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto('https://quotes.toscrape.com/search.aspx');
await page.waitForSelector('#author');
await page.select('#author', 'Albert Einstein');
await page.waitForSelector('#tag');
await page.select('#tag', 'learning');
await page.click('.btn');
await page.waitForSelector('.quote');
// extracting information from code
let quotes = await page.evaluate(() => {
let quotesElement = document.body.querySelectorAll('.quote');
let quotes = Object.values(quotesElement).map(x => {
return {
author: x.querySelector('.author').textContent ?? null,
quote: x.querySelector('.content').textContent ?? null,
tag: x.querySelector('.tag').textContent ?? null,
}
});
return quotes;
});
// logging results
console.log(quotes);
await browser.close();
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment