Skip to content

Instantly share code, notes, and snippets.

@chdorner
Created June 6, 2016 15:12
Show Gist options
  • Save chdorner/b3a42d95de21a2033695fd1148790172 to your computer and use it in GitHub Desktop.
Save chdorner/b3a42d95de21a2033695fd1148790172 to your computer and use it in GitHub Desktop.
Get destination from a redirecting URL
import requests
urls = []
with open('urls.txt', 'r') as f:
urls = f.readlines()
redirects = {}
def find_destination(url):
resp = requests.head(url)
if 'Location' in resp.headers:
return find_destination(resp.headers['Location'])
return url
for url in urls:
dest = find_destination(url)
redirects[url] = dest
with open('redirects.txt', 'w+') as f:
for old, new in redirects.iteritems():
f.write(old + '\n' + new + '\n\n')
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment