Skip to content

Instantly share code, notes, and snippets.

@fijiaaron
Last active Oct 14, 2020
Embed
What would you like to do?
Scrape Craigslist for recent car posts
requests==2.24.0
beautifulsoup4==4.9.3
# setup: pip install -r requirements.txt
import requests
from time import sleep
from bs4 import BeautifulSoup
market = 'seattle'
max = 600
start = 0
while start < max:
url = "https://{market}.craigslist.org/search/cta?s={start}".format(market=market, start=start)
# print(url)
response = requests.get(url)
# print(response.content)
soup = BeautifulSoup(response.content, 'html.parser')
# print(soup)
results = soup.find_all('li', class_='result-row')
# print(len(results))
for result in results:
date = result.find(class_='result-date')
title = result.find(class_='result-title')
price = result.find(class_='result-price')
hood = result.find(class_='result-hood')
tags = result.find(class_='result-tags')
print(date.attrs['datetime'], title.text, price.text, end="\n", sep="\t")
start = start + 120
sleep(5)
## usage: python search_craigslist.py > cars.tsv
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment