Skip to content

Instantly share code, notes, and snippets.

@tscolari
Created August 16, 2012 19:05
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save tscolari/3372713 to your computer and use it in GitHub Desktop.
Save tscolari/3372713 to your computer and use it in GitHub Desktop.
S3 multiple file uploader
#!/usr/bin/env ruby
require 'rubygems'
require 'aws/s3'
## USAGE
#
# If you use a directory as parameter, every file inside it will be uploaded.
# ex: ./s3_upload_files /system/backups/
# If you use a text file as parameter, it should have one file name per line, all files listed will be uploaded.
# ex: ./s3_upload_files changeset
#
## CONFIGURATION
credentials = {
access_key_id: "",
secret_access_key: "",
bucket_name: ""
}
# Directory path that should be excluded in S3 side of the copy
base_path = nil # turn off
# base_path = '/system/backups/'
#########################################################################
#########################################################################
#########################################################################
class S3FilesUploader
def initialize(aws_credentials)
@bucket_name = aws_credentials.delete(:bucket_name)
@credentials = aws_credentials
configure_aws
end
def upload_from_directory(directory, options = {})
options[:base_path] = directory unless options[:base_path]
Dir["#{directory}/**/*"].each do |file|
upload_file file, options
end
end
def upload_from_file_list(file_name, options = {})
File.open(file_name).each_line do |file|
upload_file file, options
end
end
private
def upload_file(file_location, options = {})
unless File.directory? file_location
print "Uploading #{file_location}..."
begin
target_name = file_location.gsub(/#{options[:base_path]}/, '')
AWS::S3::S3Object.store(target_name, open(file_location.strip), @bucket_name)
puts " ok"
rescue Exception => e
puts " fail! (#{e.message})"
end
end
end
def configure_aws
AWS::S3::Base.establish_connection!(@credentials)
end
end
s3_uploader = S3FilesUploader.new(credentials)
if File.directory? ARGV[0]
upload_type = :directory
else
upload_type = :file_name
end
case upload_type
when :file_name
s3_uploader.upload_from_file_list(ARGV[0], base_path: base_path)
when :directory
s3_uploader.upload_from_directory(ARGV[0], base_path: base_path)
end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment