Skip to content

Instantly share code, notes, and snippets.

@ychaouche
Created June 30, 2024 16:00
Show Gist options
  • Save ychaouche/90a9b976d25104603fef69ab5eb5bfff to your computer and use it in GitHub Desktop.
Save ychaouche/90a9b976d25104603fef69ab5eb5bfff to your computer and use it in GitHub Desktop.
# while no 404 or 500
while curl -f "$homepage?page=$page" > /tmp/curl.last 2>/tmp/curl.last.error
do
# add a trailing newline
echo 2>>/tmp/curl.last.error
# parse the gist
~/CODE/TEST/BASH/filegist < /tmp/curl.last | tee /tmp/curl.last.results
# if parsing didn't find anything,
# break out of the loop
if ! [[ -s /tmp/curl.last.results ]]
then
echo "nothing to see here!"
break
fi
# else, prepare for next page
((page++))
echo '#####'
echo "next page: $homepage?page=$page"
echo '#####'
sleep 0.3
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment