README.md

How to migrate ActiveStorage attachments from local (file-system) storage to S3/Amazon?

There are a couple of versions of this script floating around. My inspiration has been this great article

I went through their code and found most of it is not needed and it is a relict of days long gone. Now ActiveStorage has beautiful APIs and its all quite easy.

How to use

1. create a new service configuration for S3 named it amazon
2. backup everything!!!
3. switch your default service over to :amazon so new attachments are created there
4. run this task to copy over the blobs to S3
5. remove the old data in rails-app/storage/* (keep the backup!)

Why?

You can run this task repeatedly and it will only touch blobs that are in the wrong place (repeatable).

Using in-memory buffer (StringIO) instead of creating a milion Tempfiles (faster).

Does not delete original data so if you have a problem the files are still there and hopefully you made a backup of your db (safe).

rails_migrate_to_s3.rake

def migrate(from, to)
  puts "Current default service: #{ActiveStorage::Blob.service.name}"
  configs = Rails.configuration.active_storage.service_configurations
  from_service = ActiveStorage::Service.configure from, configs
  to_service   = ActiveStorage::Service.configure to, configs

  ActiveStorage::Blob.service = from_service

  blobs = ActiveStorage::Blob.where service_name: from
  puts "Copying #{blobs.count} blobs from #{from} to service #{to}..."
  blobs.find_each do |blob|
    print '.'
    buffer = StringIO.new
    buffer.binmode
    buffer << blob.download
    buffer.rewind
    to_service.upload(blob.key, buffer, checksum: blob.checksum)
    blob.update! service_name: to
  rescue Errno::ENOENT
    puts "Rescued by Errno::ENOENT statement. ID: #{blob.id} / Key: #{blob.key}"
  rescue ActiveStorage::FileNotFoundError
    puts "Rescued by FileNotFoundError. ID: #{blob.id} / Key: #{blob.key}"
  end
  puts "Done."
end

namespace :storage do
  desc 'Migrate ActiveStorage files from local to Amazon S3'
  task migrate: :environment do
    migrate(:local, :amazon)
  end
end