Skip to content

Instantly share code, notes, and snippets.

@sasqwatch
Forked from gwen001/otestu.sh
Created August 18, 2020 04:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save sasqwatch/809461726900e1b25173099ec84749d0 to your computer and use it in GitHub Desktop.
Save sasqwatch/809461726900e1b25173099ec84749d0 to your computer and use it in GitHub Desktop.
onliner to quickly test some urls using curl
cat plainurls.txt | while read u;do echo "$(printf '%-100s' "$u")-> $((curl -I -s -m 5 -k "$u"||echo KO)|head -n 1 -)"; done
From a json file:
cat waybackurls.json|jq -r '.[]'|grep 'http'|cut -d '"' -f 2 | while read u;do echo "$(printf '%-100s' "$u")-> $((curl -I -s -m 5 -k "$u"||echo KO)|head -n 1 -)"; done
Using parallel to speed up the process:
cat plainurls.txt | parallel -j 20 -I# 'echo "$(printf "%-100s" "#") -> $((curl -I -s -m 5 -k "#"||echo KO)|head -n 1 -)"'
function otestu {
cat $1 | parallel -j 20 -I# 'echo "$(printf "%-100s" "#") -> $((curl -I -s -m 5 -k "#"||echo KO)|head -n 1 -)"'
#while read u;do echo "$(printf '%-100s' "$u")-> $((curl -I -s -m 5 -k "$u"||echo KO)|head -n 1 -)";done < $1
}
function otestuj {
cat $1 | jq -r '.[]'|grep 'http'|cut -d '"' -f 2 | parallel -j 20 -I# 'echo "$(printf "%-100s" "#") -> $((curl -I -s -m 5 -k "#"||echo KO)|head -n 1 -)"'
#while read u;do echo "$(printf '%-100s' "$u")-> $((curl -I -s -m 5 -k "$u"||echo KO)|head -n 1 -)";done < $1
}
ALTERNATIVES:
@4_N_K_1_T
xargs -a u -I{} curl {} -s -o /dev/null -w "%{http_code}""\t -> \t""{}""\n";
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment