Skip to content

Instantly share code, notes, and snippets.

@mackermedia

mackermedia/gist:8545813

Created January 21, 2014 18:48
Show Gist options
  • Save mackermedia/8545813 to your computer and use it in GitHub Desktop.
Save mackermedia/8545813 to your computer and use it in GitHub Desktop.
Download ZIP
Craigslist Housing Parser
Raw
gistfile1.md

Looking for housing in Boulder is a bit of a nightmare. Separating the signal from the noise is a taxing chore, so I decided to come up with a little Craigslist parser to try and isolate only the information I cared about.

I wrote a parser (and accompanying web service to display results and persist records to a database) and found that Craigslist blocks requests coming from AWS servers. I then re-architected the application to have a standalone parser that runs on a Raspberry Pi to get around Craigslist blocking. This task is automated to run every few hours and pull out contextually relevant data or filter out results that I've blacklisted.

This makes looking for housing in Boudler a much more pleasant experience.

(Will consider open sourcing, however Craigslist TOS restrict parsing their data)

  • parsed results
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment