Skip to content

Instantly share code, notes, and snippets.

@csinchok
Created November 12, 2012 21:51
Show Gist options
  • Save csinchok/4062178 to your computer and use it in GitHub Desktop.
Save csinchok/4062178 to your computer and use it in GitHub Desktop.
Stats for the Million Dollar Homepage links
import requests
from bs4 import BeautifulSoup
homepage = requests.get('http://www.milliondollarhomepage.com')
soup = BeautifulSoup(homepage.content)
response_codes = {'error': 0}
urls = {}
for area in soup.find_all('area'):
href = area.get('href')
if href in urls:
continue
try:
response = requests.head(href, timeout=1.0)
urls[href] = response.status_code
if response.status_code in response_codes:
response_codes[response.status_code] += 1
else:
response_codes[response.status_code] = 0
except Exception as e:
urls[href] = str(e)
response_codes['error'] += 1
print(urls[href])
print(response_codes)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment