Rails 5.2.3: Cache Active Storage Blobs and Variants through Cloudfront

readme.md

Basic Usage

# ENV["ASSETS_HOST"] = 'thing.cloudfront.net'
class Foo < ApplicationRecord
  has_one_attached :thing
end

foo = Foo.first

# https://thing.cloudfront.net/rails/active_storage/blobs/etc..etc
proxy_url(foo.image)

# https://thing.cloudfront.net/rails/active_storage/representations/etc..etc
proxy_url(foo.image.variant(resize: '100x100'))

This is nowhere near perfect. Just gets the job done in a simple manner. Inspiration from https://github.com/rails/rails/pull/34477

application_helper.rb

# app/helpers/application_helper.rb
module ApplicationHelper
  # active_storage_item could be a blob or variant object
  def proxy_url(active_storage_item, options = {})
    options.merge!(host: ENV['ASSETS_HOST']) if ENV['ASSETS_HOST'].present?

    # proxy: 'true' allows you to stil have the original functionality while
    # being able to proxy through a CDN. You've got to ensure that your CDN
    # forwards this param otherwise active storage will always do the default
    # behavior which is a redirect to the service.
    # This is also meant to be intended for items that are always going to be
    # public. If you have files that might be private then there's no point in
    # caching it with cloudfront.
    # See lib/core_extensions/active_storage/blob/downloader and
    # lib/core_extensions/active_storage/representation/downloader
    polymorphic_url(active_storage_item, options.merge(proxy: 'true'))
  end
end

blob_downloader.rb

# lib/core_extensions/active_storage/blob/downloader.rb
# had to make the file a different name because reasons ¯\_(ツ)_/¯
module CoreExtensions
  module ActiveStorage
    module Blob
      module Downloader
        include CoreExtensions::ActiveStorage::Headers

        def show
          return super unless params[:proxy] == 'true'
          set_headers(@blob)
          @blob.download do |chunk|
            response.stream.write(chunk)
          end
        ensure
          response.stream.close
        end
      end
    end
  end
end

core_extentions.rb

# config/initializers/core_extentions.rb
Dir[File.expand_path(File.join(File.dirname(File.absolute_path(__FILE__)), 'lib/core_extensions')) + "/**/*.rb"].each do |file|
  require file
end
ActiveStorage::RepresentationsController.prepend CoreExtensions::ActiveStorage::Representation::Downloader
ActiveStorage::BlobsController.prepend CoreExtensions::ActiveStorage::Blob::Downloader

headers.rb

# lib/core_extensions/active_storage/headers.rb
module CoreExtensions
  module ActiveStorage
    module Headers
      extend ActiveSupport::Concern

      private

        def set_headers(blob)
          # Hard coded this to 365, simply because thats what I want tbh
          expires_in 365.days, public: true

          response.headers["Content-Type"] = blob.content_type
          # Commented this out because in Rails 5.2.3 this isn't a thing
          # response.headers["Content-Disposition"] = ActionDispatch::Http::ContentDisposition.format(
          #   disposition: params[:disposition] || "inline",
          #   filename: blob.filename.sanitized
          # )
        end
    end
  end
end

representation_downloader.rb

# lib/core_extensions/active_storage/representation/downloader.rb
# had to make the file a different name because reasons ¯\_(ツ)_/¯
module CoreExtensions
  module ActiveStorage
    module Representation
      module Downloader
        include CoreExtensions::ActiveStorage::Headers

        def show
          return super unless params[:proxy] == 'true'
          set_headers(@blob)
          variant = @blob.representation(params[:variation_key]).processed
          @blob.service.download(variant.key) do |chunk|
            response.stream.write(chunk)
          end
        ensure
          response.stream.close
        end
      end
    end
  end
end

I'm getting NameError: uninitialized constant CoreExtensions

In line 2 of core_extensions.rb, I had to change the Dir path to: "#{Rails.root}/lib/core_extensions/**/*.rb"

In line 2 of core_extensions.rb, I had to change the Dir path to: "#{Rails.root}/lib/core_extensions/**/*.rb"

Could you show me your full file? Now I'm getting NameError: uninitialized constant CoreExtensions::ActiveStorage::Headers

I'm using rails 6

I also had to put the files in app/lib rather than at the root /lib. Since Rails wasn't autoloading before boot in production, it was giving me NameError.

Still couldn't get Cloudfront to work with variants for some reason, so I ended up scrapping the monkey patch and leveraged ImageKit (referral link 🙏🏽), which is actually pretty dope. Easy to work with since it plugs directly into S3 and handles all the variant and cropping I was looking to do with ActiveRecord. I'm no longer using ActiveRecord variants.

Then in the API response, instead of using url_for(image), I created an application helper that builds a unique url for ImageKit based on the blob key, so it hits S3 directly and bypasses the server.

Hey @ArthurTruong5,

I'm sorry for the late reply, I missed these comments in my mailbox.

I realised that in the application that I've been working on, a previous developer added this line to the application.rb and hence why I didn't run into this problem.

config.eager_load_paths += %W(#{config.root}/lib)

Did you come right with this problem or is it still throwing a NameError?

Hey @joelrojo,

Thats a little strange that it didn't work for variants for you.
When I created this monkey patch that was the first thing I was testing with.

What was happening when you were trying to implement it with variants?
Was it still hitting your server and redirecting to s3?

Hi

I have tried this and I was able successfully to get this to work. Though my cloud front setup is having issues (which I'm resolving), how you plan to handle the existing objects in S3 with a new CloudFront service in place. Do we need to migrate things or something?

hey @Grey-worm,

Could you elaborate a little more in regards with existing objects?
I'm not following here

Hi @howler,

Ideally ActiveStorage blob will provide us the s3 object key to search at S3 to retrieve/generate the URL. Now that we pointed the storage location to Cloudfront host configured on the bucket, Do we need to configure any thing additionally at cloudfront side(I mean any policies or OAI settings) or will it simply fetch from cloudfront.

Note: I'm new to ActiveStorage, correct me if I didn't convey any thing in right way.

hey @Grey-worm,

If you comment out the code in core_extentions.rb and you call url_for on an active storage object (blob or variant), what is the url that you end up with?

Is it something like bucket-name.s3-region.amazonaws.com or xxxx.cloudfront.net?

Without core_extentions.rb snippet in place

url_for: /rails/active_storage/..... [redirects to http://bucket-name.xxx.cloudfront.net/key/...]
via proxy_url method: xxx.cloudfront.net:3000/rails/active_storage/...
object.image.service_url: bucket-name.xxx.cloudfront.net/key/...

@Grey-worm okay cool.

I'm not really seeing the need for you to put this in place then.
This monkey patch would be used in the instance where you have CDN in front of your rails application.

Okay

In application_helper.rb, the second parameter options of the method proxy_url should have default value {} like,

# on above
def proxy_url(active_storage_item, options)
end

# should be
def proxy_url(active_storage_item, options = {})
end

To proxy_url(attachable) work.

Hey @yhk1038,
You're definitely correct, I'll make the change. 👍

Any tips on how to handle this in rails v6.1?

I added this to my production.rb

  config.action_controller.asset_host = ENV['CLOUDFRONT']
  config.active_storage.service = :amazon
  config.active_storage.delivery_method = :proxy

And then tweaked your helper to

  def proxy_url(active_storage_item, options = {})
    options.merge!(host: ENV['CLOUDFRONT']) if ENV['CLOUDFRONT'].present?
    polymorphic_url(active_storage_item, options)
  end

I don't think this is needed anymore, at least not from rails 6.1 and up. Proxying was added to ActiveStorage.

More details here:
rails/rails#34477
https://github.com/rails/rails/blob/main/activestorage/README.md#proxying
https://guides.rubyonrails.org/configuring.html#configuring-active-storage