Skip to content

Instantly share code, notes, and snippets.

@gwen001
Created December 5, 2019 14:15
Show Gist options
  • Star 8 You must be signed in to star a gist
  • Fork 5 You must be signed in to fork a gist
  • Save gwen001/fcea0edc616be58df9a087fb5f02fb18 to your computer and use it in GitHub Desktop.
Save gwen001/fcea0edc616be58df9a087fb5f02fb18 to your computer and use it in GitHub Desktop.
onliner to extract params from url
echo "https://www.example.com/?aaa=bbb&ccc=ddd" | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[1]}}'
while read u; do echo $u | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[1]}}'; done < plainurls.txt | sort -fu
From wayback json file:
cat waybackurls.json|jq -r '.[]'|grep 'http'|cut -d '"' -f 2 | while read u; do echo $u | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[1]}}'; done | sort -fu
function oparam {
echo $1 | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[1]}}'
}
function oparamv {
echo $1 | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[2]}}'
}
function oparams {
while read u; do echo $u | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[1]}}'; done < $1 | sort -fu
}
function oparamsv {
while read u; do echo $u | tr '?' '&' | awk -F '&' '{for(i=2;i<=NF;i++){split($i,t,"=");print t[2]}}'; done < $1 | sort -fu
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment