Skip to content

Instantly share code, notes, and snippets.

@nikosvaggalis
Created November 23, 2017 11:57
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save nikosvaggalis/05198cbf041f5e89358d55b3693ebc6f to your computer and use it in GitHub Desktop.
Save nikosvaggalis/05198cbf041f5e89358d55b3693ebc6f to your computer and use it in GitHub Desktop.
Puppeteer - Scrape smadeseek.com 2 levels deep
const puppeteer = require("puppeteer");
const url = "http://smadeseek.com/smartphones";
const imgSelector = "#contentx > div > div img";
const tableSelector = "#masthead";
const tableSelector2 = "#masthead > div > div:nth-child(2) > div > div > div.col-md-6.col-sm-6 > table:nth-child(2)";
(async () => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto(url, { waitUntil: ["load","domcontentloaded"] });
await page.waitForSelector(imgSelector);
const imgAll = await page.$$eval(imgSelector, images => {return images.map((image)=>image.src)});
await console.log(imgAll);
await page.click("#contentx > div > div:nth-child(2) > a > img ");
await page.waitForSelector(tableSelector);
const tableAll = await page.$eval(tableSelector2, images => images.innerHTML);
await console.log(tableAll);
await browser.close();
})();
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment