This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
####################################################################### | |
# Setup: | |
# $ python3 -m venv venv | |
# $ source venv/bin/activate | |
# $ pip install requests | |
# Usage: | |
# $ python pgatour.py | |
####################################################################### | |
import os |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
####################################################################### | |
# Setup: | |
# python3 -m venv venv | |
# source venv/bin/activate | |
# pip install lxml requests beautifulsoup4 | |
# | |
# Usage: | |
# python eygbl.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* $ npm init -y | |
* $ npm install puppeteer --save | |
* $ node sheahomes.js | |
*/ | |
const puppeteer = require('puppeteer'); | |
const url = 'https://www.sheahomes.com/new-homes/colorado/denver-area/parker/stonewalk-at-stepping-stone/'; | |
async function getLot(page, url) { | |
await page.goto(url); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import re | |
import json | |
import urlparse | |
import requests | |
from bs4 import BeautifulSoup | |
class SheaHomesScraper(object): | |
def __init__(self): | |
self.url = 'https://www.sheahomes.com/new-homes/colorado/denver-area/parker/stonewalk-at-stepping-stone/' |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import json | |
import requests | |
class LennarScraper(object): | |
def __init__(self): | |
self.url = 'https://www.lennar.com/Services/Rest/SearchMethods.svc/GetInventoryTabDetails' | |
self.session = requests.Session() | |
self.data = { | |
"CommunityID":"4531", | |
"pageState":{ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Companion code for article at http://toddhayton.com/2019/02/12/revisiting-taleo-with-puppeteer/ | |
* | |
* Setup: | |
* $ mkdir scraper/ | |
* $ cd scraper/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Companion code for article at http://toddhayton.com/... | |
* | |
* Setup: | |
* $ mkdir scraper/ | |
* $ cd scraper/ | |
* $ npm init -y | |
* $ npm install puppeteer --save | |
* $ npm install node-fetch --save | |
* $ npm install progress-stream --save |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
import json | |
import requests | |
from bs4 import BeautifulSoup | |
class L3ComScraper(object): | |
def __init__(self): | |
self.session = requests.Session() |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
node_modules/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
/* | |
* Companion code for article at http://toddhayton.com/2018/08/01/scraping-with-puppeteer/ | |
* | |
* Setup: | |
* $ mkdir scraper/ | |
* $ cd scraper/ | |
* $ npm init -y | |
* $ npm install puppeteer --save | |
* | |
* Usage: |
NewerOlder