Skip to content

Instantly share code, notes, and snippets.

@gboone
Created November 14, 2014 17:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gboone/6433de78fd067d18c080 to your computer and use it in GitHub Desktop.
Save gboone/6433de78fd067d18c080 to your computer and use it in GitHub Desktop.
Download files from a yaml file
##
# Where yaml file looks like:
# ---
# directory:
# - url
# directory2:
# - url
# ---
# Script will create the directories if they need creating
# and download the resource at each url into that directory
#
# Ruby is fun
##
require 'yaml'
require 'fileutils'
require 'curb'
ARGV.each do |a|
dir = Dir.pwd()+"/"
path = dir+a
if File.exists?(path)
file = File.read(path)
yaml = YAML.load(file)
for y in yaml
FileUtils::mkdir_p dir+y[0]
Dir.chdir(dir+y[0])
for url in y[1]
c = Curl::Easy.perform(url)
filename = c.url.split(/\?/).first.split(/\//).last
File.open(filename, 'wb') do |f|
c.on_body {|data| f << data; data.size }
c.perform
puts "=> '#{filename}'"
end
end
end
end
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment