Skip to content

Instantly share code, notes, and snippets.

Created February 2, 2013 10:54
Show Gist options
  • Save anonymous/4696874 to your computer and use it in GitHub Desktop.
Save anonymous/4696874 to your computer and use it in GitHub Desktop.
lca2013 video leecher
#!/bin/bash
# Script to leech videos from lca2013
# Will not re-download stuff you already have
#
# Requires curl, sed and grep
#
# Yes, this parses HTML with regex. Deal with it.
#
# Known bugs:
# * Does not handle Ctrl+C very well, you will end up with partial videos
#
# Wishlist:
# * It would be nice to be able to cherry pick (or exclude?) videos rather
# than just leeching the whole mirror (especially for the mirrors).
download_to=.
source_url=http://mirror.linux.org.au/linux.conf.au/2013/
rate_limit=20000 # in bytes/s
err() {
echo $1
exit 1
}
[[ -d ${download_to} ]] || \
err "Sorry, directory '${download_to}' does not exist"
curl "${source_url}" | grep href=.*ogv | sed 's/^.*href="\([^>]*\)".*$/\1/' |
while read file; do
[[ -f $file ]] || \
curl \
--limit-rate ${rate_limit} \
-o "${download_to}/${file}" \
"${source_url}/${file}"
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment