Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Web scraper that downloads archived episodes of 'Trance Around The World' podcast
#!/usr/bin/env ruby
#
# This is a web scraper which downloads mp3 files from the TATW web archives.
#
# Usage: run this script and pass it an episode range to download via arguments
#
# Dependencies: requires 'aria2' to be installed (uses the CLI tool 'aria2c')
#
require 'pathname'
require 'colorize' # Use of this gem is optional; it makes the output prettier
PLAYER_URL = 'http://www.trancearoundtheworld.com/player/play.php?id='
DOWNLOAD_URL = 'http://static.trancearoundtheworld.com/archives'
LOCAL_DOWNLOAD_DEST = '~/Downloads'
CMD = 'aria2c'
Dir.chdir Pathname(LOCAL_DOWNLOAD_DEST).expand_path
# Download a range of TATW episodes using the Aria2 CLI
unless ARGV[0] && ARGV[1]
abort 'ERROR: Supply episode range (two successive numbers) as parameters to the script, e.g. 430 450'
end
SETS = Range.new ARGV[0], ARGV[1]
puts "Attempting to download #{SETS.count} TATW episodes!"
# Episodes take about 3-5 minutes each to download with standard cable
SETS.each do |ep|
if `#{CMD} #{DOWNLOAD_URL}/TATW#{ep}.mp3`
puts "...Episode #{ep.to_s.blue} was successfully downloaded!" if File.exist?("#{Dir.pwd}/TATW#{ep}.mp3")
else
puts '...Failed '.red + 'to download episode '.yellow + ep.to_s.light_red
end
end
@SteveBenner

This comment has been minimized.

Copy link
Owner Author

@SteveBenner SteveBenner commented May 17, 2015

Instructions:

  • The script requires execution privileges to run. You can make it executable for your user with: chmod u+x unbrew.rb.
  • You can run the script by executing it normally on the command line, which means either using the absolute path, e.g. /Users/you/dl-tatw.rb, or if the file is in your current working directory, you should use: ./dl-tatw.rb.

This script requires aria2 to be installed. If you use Homebrew, it’s as simple as brew install aria2.

@doctorwho42

This comment has been minimized.

Copy link

@doctorwho42 doctorwho42 commented Sep 27, 2018

Hey Steve, I was wondering if you have a TATW archive I could download from you. I was in the process of writing a script like this until I found the TATW archive missing due to web-address changing or Above and beyond hiding their TATW archives since they are into their new set Group Therapy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.