paulmwatson/cdxt_commoncrawl_example_domain.sh

Created May 25, 2021 10:46

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/paulmwatson/7a1b30312f3643b0ba4ab15833735835.js"></script>
Save paulmwatson/7a1b30312f3643b0ba4ab15833735835 to your computer and use it in GitHub Desktop.

Download ZIP

Use CDXT Toolkit to search CommonCrawl for certain domains/URLs

Raw

cdxt_commoncrawl_example_domain.sh

cdxt --cc --limit 100 --filter '=status:200' iter 'github.com/*'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment