Skip to content

Instantly share code, notes, and snippets.

@kevmo
Created December 12, 2014 20:54
Show Gist options
  • Save kevmo/513b5d34c8ed74e44334 to your computer and use it in GitHub Desktop.
Save kevmo/513b5d34c8ed74e44334 to your computer and use it in GitHub Desktop.
Scrape a page for images using BeautifulSoup
import sys
import urllib2
import contextlib
from bs4 import BeautifulSoup
if len(sys.argv) < 2:
print 'Error: Please provide a URL to scrape.'
sys.exit(1)
URL = sys.argv[1]
with contextlib.closing(urllib2.urlopen(URL)) as r:
html = r.read()
soup = BeautifulSoup(html)
og_image = soup.find('meta', attrs={'property': 'og:image'})
if og_image:
print og_image.get('content')
else:
for img in soup.find_all('img'):
print img.get('src')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment