Skip to content

Instantly share code, notes, and snippets.

@hjst
Created July 7, 2017 17:08
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hjst/8e90681c5f22188b0ceadd5c5eebef00 to your computer and use it in GitHub Desktop.
Save hjst/8e90681c5f22188b0ceadd5c5eebef00 to your computer and use it in GitHub Desktop.
Try to find valid downloads for files with a version number in their URL
curl --head --silent --output /dev/null \
--write-out "%{http_code}: %{url_effective}\n" \
https://s3.amazonaws.com/bucket/foo/bar-2.[6-82].[0-9]-[1-9].baz.[0-9].tgz \
| grep -v ^403
# --head # do a HEAD, not a GET; understand the tradeoffs
# --silent # hush the usual output progress bars etc.
# --output /dev/null # hush the remaining output (header echoes)
# --write-out "%{http_code}: %{url_effective}\n" # output these specific fields on a line, suitable for grep
# .../bucket/foo/bar-2.[6-82].[0-9]-[1-9].baz.[0-9].tgz # URL pattern, usual bash syntax
# | grep -v ^403 # output ONLY lines that aren't 403 (all the "misses" were 403 in this case;
# # I wanted to catch both 200s and possible 30x redirects)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment