Skip to content

Instantly share code, notes, and snippets.

@akamhy

akamhy/example_cdx.py

Last active January 5, 2021 04:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save akamhy/bcfa08659a9aae335542cae31bc0c9be to your computer and use it in GitHub Desktop.
Save akamhy/bcfa08659a9aae335542cae31bc0c9be to your computer and use it in GitHub Desktop.
Download ZIP
Cdx Pagination API, wildcard search
Raw
example_cdx.py
from waybackpy import Cdx # waybackpy >= 2.3.3
url = "https://www8.gsb.columbia.edu/cbs-directory/detail/*"
user_agent = "Your-apps-user-agent"
cdx = Cdx(url=url, user_agent=user_agent)
snapshots = cdx.snapshots()
for snapshot in snapshots:
print(snapshot.archive_url)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment