Skip to content

Instantly share code, notes, and snippets.

@crevulus
Last active August 16, 2022 20:10
Show Gist options
  • Save crevulus/76db176b26ce4fab29889a8960aa007f to your computer and use it in GitHub Desktop.
Save crevulus/76db176b26ce4fab29889a8960aa007f to your computer and use it in GitHub Desktop.
A small script that uses puppeteer to scrape craigslist and return images for a search term
const puppeteer = require("puppeteer");
async function getImages(searchTerm) {
const browser = await puppeteer.launch({
// headless: false,
defaultViewport: null,
});
const page = await browser.newPage();
const url = `https://nh.craigslist.org/d/for-sale/search/sss?query=${searchTerm}&sort=rel&hasPic=1`;
await page.goto(url);
await page.waitForSelector(".result-row");
const results = await page.$$eval(".result-row", (rows) => {
return rows.map((row) => {
const properties = {};
const heading = row.querySelector(".result-info .result-heading");
properties.heading = heading ? heading.innerText : "";
const price = row.querySelector(".result-info .result-price");
properties.price = price ? price.innerText : "";
const postLink = row.querySelector(".result-image").getAttribute("href");
properties.link = postLink;
const image = row.querySelector(
'.result-image .swipe .swipe-wrap [data-index="0"] img'
);
properties.image = image ? image.src : "";
return properties;
});
});
console.log(results);
return results;
}
getImages("car");
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment