Skip to content

Instantly share code, notes, and snippets.

@bf4
Last active November 11, 2017 22:18
  • Star 5 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save bf4/4070991 to your computer and use it in GitHub Desktop.
Download my destroyallsoftware videos

This script will download all ios files in the current directory.

It will check for already downloaded files in the current directory.

It can work interactively or with a config.yml (copy from config.yml.template)

usage

das = DasDownloader.new
das.sign_in.download_videos

or

ruby das.rb

Any idea how to TDD this?

Other comments?

Benjamin Fleischer Do what you want with this code

my_email: bobs@initech.com
my_password: doinggreat
# downloads all videos to the current directory
# obviously you must be a paying member
# works for me, rails 1.9.3, mechanize 2.5.1, osx mountain lion
require 'yaml'
require 'mechanize'
class DasDownloader
attr_reader :agent, :current_page, :my_email, :my_password, :config
def initialize
initialize_agent
puts "initalized"
end
def sign_in
puts "signing in"
signin_url = "https://www.destroyallsoftware.com/screencasts/users/sign_in"
@current_page = page = get_url(signin_url)
form = page.form
set_credentials
form.field_with('user[email]').value = my_email
form.field_with('user[password]').value = my_password
form.checkbox_with('user[remember_me]').check
@current_page = agent.submit(form, form.buttons.first)
self
end
def download_videos
puts "downloading videos"
download_links.each do |link|
download_video(link)
end
end
private
def initialize_agent
@agent = Mechanize.new
agent.user_agent = 'Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2) Gecko/20100115 Firefox/3.6' # not sure what user_agent_alias I can set
end
def get_url(url, parameters=[], headers={})
referer = "https://www.destroyallsoftware.com/screencasts/catalog"
agent.get(url, parameters, referer, headers) # weird api, made explicit here
end
def download_links
existing_videos = Dir.glob('*.mov')
all_links = current_page.links_with(:href => %r{download_ios})
old_links, new_links = all_links.partition do |link|
regexp = Regexp.new(link_slug(link))
existing_videos.any? {|mov| mov =~ regexp }
end
if new_links.size.zero?
puts "everything is downloaded"
else
puts "already downloaded \n\t#{old_links.map(&:href).join("\n\t")}"
puts "downloading \n\t#{new_links.map(&:href).join("\n\t")}"
end
new_links
end
def link_slug(link)
link.href.split('/')[-2]
end
def download_video(link)
puts
print "downloading #{link_slug(link)} (into memory? yuck)... "
file = link.click
name = file.filename[/[\w-]+/] + '.mov' #regex isn't working right here. I don't care why at the moment. TODO FIXME
print "saving #{name}... "
file.save name
puts "success"
end
def set_credentials
if valid_config?
@my_email = config['my_email']
@my_password = config['my_password']
else
puts "enter email"
@my_email = gets.chomp
puts "enter password"
@my_password = gets.chomp
end
end
def valid_config?
config_file = File.join(File.dirname(__FILE__),'config.yml')
if File.exists?(config_file)
@config = YAML::load(File.read(config_file))
['my_email','my_password'].all? {|key| config[key].to_s.size > 2 }
else
false
end
end
end
if __FILE__ == $0
das = DasDownloader.new
das.sign_in.download_videos
end
# A sample Gemfile
source "https://rubygems.org"
gem 'mechanize'
GEM
remote: https://rubygems.org/
specs:
domain_name (0.5.4)
unf (~> 0.0.3)
mechanize (2.5.1)
domain_name (~> 0.5, >= 0.5.1)
mime-types (~> 1.17, >= 1.17.2)
net-http-digest_auth (~> 1.1, >= 1.1.1)
net-http-persistent (~> 2.5, >= 2.5.2)
nokogiri (~> 1.4)
ntlm-http (~> 0.1, >= 0.1.1)
webrobots (~> 0.0, >= 0.0.9)
mime-types (1.19)
net-http-digest_auth (1.2.1)
net-http-persistent (2.8)
nokogiri (1.5.5)
ntlm-http (0.1.1)
unf (0.0.5)
unf_ext
unf_ext (0.0.5)
webrobots (0.0.13)
PLATFORMS
ruby
DEPENDENCIES
mechanize
@andymatuschak
Copy link

This was helpful; thank you.

@bf4
Copy link
Author

bf4 commented Dec 31, 2012

@andymatuschak glad it helped. :)

@bf4
Copy link
Author

bf4 commented Feb 13, 2013

fyi, there's another one that's less rubyish https://gist.github.com/orlando/1555684 and also this repo https://github.com/bemurphy/das_catalog

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment