Skip to content

Instantly share code, notes, and snippets.

@shaneshifflett
Created January 8, 2013 19:13
Show Gist options
  • Save shaneshifflett/4486946 to your computer and use it in GitHub Desktop.
Save shaneshifflett/4486946 to your computer and use it in GitHub Desktop.
download files from a list of URLs
import requests
scraper_urls = [
'http://foo.net/TESTDIR/TEST.pdf',
'http://foo.net/TESTDIR/TEST2.pdf'
]
p_session = requests.session()
for surl in scraper_urls:
content = p_session.get(surl).content
fname = surl.split('/')[-1]
with open(fname, 'w') as f:
f.write(content)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment