Skip to content

Instantly share code, notes, and snippets.

@dwurf
Forked from anonymous/leech.sh
Last active December 12, 2015 02:18
Show Gist options
  • Save dwurf/4696987 to your computer and use it in GitHub Desktop.
Save dwurf/4696987 to your computer and use it in GitHub Desktop.
#!/bin/bash
# Script to leech videos from lca2013
# Will not re-download stuff you already have
#
# Requires curl, sed and grep
#
# Yes, this parses HTML with regex. Deal with it.
#
# Known bugs:
# * Does not handle Ctrl+C very well, you will end up with partial videos
# (re-running the script will now resume partial videos)
#
# Wishlist:
# * It would be nice to be able to cherry pick (or exclude?) videos rather
# than just leeching the whole mirror (especially for the mirrors).
download_to=.
source_url=http://mirror.linux.org.au/linux.conf.au/2013/
rate_limit=20000 # in bytes/s
err() {
echo $1
exit 1
}
[[ -d ${download_to} ]] || \
err "Sorry, directory '${download_to}' does not exist"
curl "${source_url}" | grep href=.*ogv | sed 's/^.*href="\([^>]*\)".*$/\1/' |
while read file; do
curl \
--limit-rate ${rate_limit} \
-C - \
-o "${download_to}/${file}" \
"${source_url}/${file}"
done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment