Skip to content

Instantly share code, notes, and snippets.

View chrishham's full-sized avatar

Christopher Chamaletsos chrishham

View GitHub Profile
@chrishham
chrishham / sample.js
Last active April 18, 2021 15:59
Super Web Scraper - Puppeteer Script
// Available functions :
//
// page : https://pptr.dev/#?product=Puppeteer&version=v8.0.0&show=api-class-page
// Custom
// scrollToBottom
// scrollToSelector
await scrollToBottom(page);
await page.waitForSelector('iframe#dsq-app2953')
await scrollToBottom(page);
@chrishham
chrishham / sample.js
Last active April 19, 2021 16:45
Super Web Scraper - Sample node-html-parser code
// https://www.npmjs.com/package/node-html-parser
// htmlRoot gets injected and is available to your script
// Helper Functions:
// select:removeWhitespace().text & el.attributes.content
// selectAll: same as above
const result = []
const list = Array.from(htmlRoot.querySelectorAll(':is(.AdvItemBox,.FreeListingItemBox)'))
for (const el of list) {
const isAd = !!el.querySelector('.AdvDetailsArea')