Skip to content

Instantly share code, notes, and snippets.

@hatemhosny
Forked from bmorelli25/allScrape.js
Created November 7, 2017 23:36
Show Gist options
  • Save hatemhosny/6d5242807f059f2d369f5b853b5783c6 to your computer and use it in GitHub Desktop.
Save hatemhosny/6d5242807f059f2d369f5b853b5783c6 to your computer and use it in GitHub Desktop.
Scrape all books on the hompage
const puppeteer = require('puppeteer');
let scrape = async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();
await page.goto('http://books.toscrape.com/');
const result = await page.evaluate(() => {
let data = []; // Create an empty array that will store our data
let elements = document.querySelectorAll('.product_pod'); // Select all Products
for (var element of elements){ // Loop through each proudct
let title = element.childNodes[5].innerText; // Select the title
let price = element.childNodes[7].children[0].innerText; // Select the price
data.push({title, price}); // Push an object with the data onto our array
}
return data; // Return our data array
});
browser.close();
return result; // Return the data
};
scrape().then((value) => {
console.log(value); // Success!
});
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment