Skip to content

Instantly share code, notes, and snippets.

@ward
Created July 15, 2019 13:35
Show Gist options
  • Save ward/aef829111610ecf0a26aefdebd3ce69f to your computer and use it in GitHub Desktop.
Save ward/aef829111610ecf0a26aefdebd3ce69f to your computer and use it in GitHub Desktop.
Parse a open.kattis.com problem listing page to fetch the individual problems for offline use.
#!/bin/sh
# Expects a problems.txt to exist that is the source code of a problems page
if [ ! -f problems.txt ]; then
echo "You should get the source of a page with a list of problems. Save it in problems.txt."
exit 1
fi
problems=$(sed '/problems\//!d' problems.txt | sed '/\/submit/d' | sed '/\/statistics/d' | sed 's/^.*problems\/\(.*\)".*$/\1/')
mkdir -p problems
for problem in $problems; do
if [ ! -f "problems/$problem.html" ]; then
wget -nv -O "problems/$problem.html" "https://open.kattis.com/problems/$problem"
sleep 1
else
echo "Skipping $problem. We already have that one."
fi
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment