Skip to content

Instantly share code, notes, and snippets.

View sheahomes.js
/*
* $ npm init -y
* $ npm install puppeteer --save
* $ node sheahomes.js
*/
const puppeteer = require('puppeteer');
const url = 'https://www.sheahomes.com/new-homes/colorado/denver-area/parker/stonewalk-at-stepping-stone/';
async function getLot(page, url) {
await page.goto(url);
View sheahomes.py
import re
import json
import urlparse
import requests
from bs4 import BeautifulSoup
class SheaHomesScraper(object):
def __init__(self):
self.url = 'https://www.sheahomes.com/new-homes/colorado/denver-area/parker/stonewalk-at-stepping-stone/'
View lennar.py
import json
import requests
class LennarScraper(object):
def __init__(self):
self.url = 'https://www.lennar.com/Services/Rest/SearchMethods.svc/GetInventoryTabDetails'
self.session = requests.Session()
self.data = {
"CommunityID":"4531",
"pageState":{
View l3com_scraper.js
/*
* Companion code for article at http://toddhayton.com/2019/02/12/revisiting-taleo-with-puppeteer/
*
* Setup:
* $ mkdir scraper/
* $ cd scraper/
View campaignfinancemd.js
/*
* Companion code for article at http://toddhayton.com/...
*
* Setup:
* $ mkdir scraper/
* $ cd scraper/
* $ npm init -y
* $ npm install puppeteer --save
* $ npm install node-fetch --save
* $ npm install progress-stream --save
View l3com_scraper.py
#!/usr/bin/env python
import json
import requests
from bs4 import BeautifulSoup
class L3ComScraper(object):
def __init__(self):
self.session = requests.Session()
View rid_scraper.js
/*
* Companion code for article at http://toddhayton.com/2018/08/01/scraping-with-puppeteer/
*
* Setup:
* $ mkdir scraper/
* $ cd scraper/
* $ npm init -y
* $ npm install puppeteer --save
*
* Usage:
View gist:76ee55882eaf26c8a0de6a1a01ef023e
#!/usr/bin/env python
import time
import json
import requests
class Scraper(object):
def __init__(self):
self.url = 'http://preowned.ferrari.com/vdata'
self.session = requests.Session()
View keybase.md

Keybase proof

I hereby claim:

  • I am thayton on github.
  • I am thayton (https://keybase.io/thayton) on keybase.
  • I have a public key ASDwUwmXo5-1N9a9CTNzTATM_EndqNJMp3O_hzTBhaLOmQo

To claim this, I am signing this object:

You can’t perform that action at this time.