Skip to content

Instantly share code, notes, and snippets.

@DDRBoxman
Created February 16, 2016 05:57
Show Gist options
  • Save DDRBoxman/819c44c6aca3243e5fde to your computer and use it in GitHub Desktop.
Save DDRBoxman/819c44c6aca3243e5fde to your computer and use it in GitHub Desktop.
Get the reps in Texas
import requests
from bs4 import BeautifulSoup
import unicodecsv as csv
r = requests.get("https://www.txdirectory.com/online/txsenate/")
c = r.content
soup = BeautifulSoup(c, "html.parser")
reps = []
for td in soup.select("td[width=160]"):
if (td.a == None):
break
reps.append([td.a.text, td.small.text[1:2], td.parent.select("td[width=25]")[0].text, td.parent.select("td[width=50]")[0].text])
resultFile = open("output.csv",'wb')
wr = csv.writer(resultFile, dialect='excel')
wr.writerows(reps)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment