Skip to content

Instantly share code, notes, and snippets.

@spencerkittleson
Last active March 17, 2023 21:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save spencerkittleson/5a098baa1319563d406831a973f0888c to your computer and use it in GitHub Desktop.
Save spencerkittleson/5a098baa1319563d406831a973f0888c to your computer and use it in GitHub Desktop.
Bot to get titles
import puppeteer from "puppeteer";
(async () => {
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();
await page.goto(
"https://sfbay.craigslist.org/search/sby/sof#search=1~thumb~0~0"
);
// Set screen size
await page.setViewport({ width: 1080, height: 1024 });
// Wait and click on first result
const searchResultSelector = ".titlestring";
await page.waitForSelector(searchResultSelector);
// https://github.com/puppeteer/puppeteer/issues/3713#issuecomment-452929764
const titleLinks = await page.$$(searchResultSelector);
for (const titleLink of titleLinks) {
const text = await titleLink.evaluate((el) => el.textContent);
const href = await titleLink.evaluate((el) => el.href);
if (text.indexOf("Python") != -1) {
console.log(href);
}
}
await browser.close();
})();
{
"name": "botclist",
"version": "1.0.0",
"description": "",
"type": "module",
"main": "index.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "ISC",
"devDependencies": {
"puppeteer": "^19.7.5"
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment