Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Rails 5.2.3: Cache Active Storage Blobs and Variants through Cloudfront

Basic Usage

# ENV["ASSETS_HOST"] = 'thing.cloudfront.net'
class Foo < ApplicationRecord
  has_one_attached :thing
end

foo = Foo.first

# https://thing.cloudfront.net/rails/active_storage/blobs/etc..etc
proxy_url(foo.image)

# https://thing.cloudfront.net/rails/active_storage/representations/etc..etc
proxy_url(foo.image.variant(resize: '100x100'))

This is nowhere near perfect. Just gets the job done in a simple manner. Inspiration from https://github.com/rails/rails/pull/34477

# app/helpers/application_helper.rb
module ApplicationHelper
# active_storage_item could be a blob or variant object
def proxy_url(active_storage_item, options = {})
options.merge!(host: ENV['ASSETS_HOST']) if ENV['ASSETS_HOST'].present?
# proxy: 'true' allows you to stil have the original functionality while
# being able to proxy through a CDN. You've got to ensure that your CDN
# forwards this param otherwise active storage will always do the default
# behavior which is a redirect to the service.
# This is also meant to be intended for items that are always going to be
# public. If you have files that might be private then there's no point in
# caching it with cloudfront.
# See lib/core_extensions/active_storage/blob/downloader and
# lib/core_extensions/active_storage/representation/downloader
polymorphic_url(active_storage_item, options.merge(proxy: 'true'))
end
end
# lib/core_extensions/active_storage/blob/downloader.rb
# had to make the file a different name because reasons ¯\_(ツ)_/¯
module CoreExtensions
module ActiveStorage
module Blob
module Downloader
include CoreExtensions::ActiveStorage::Headers
def show
return super unless params[:proxy] == 'true'
set_headers(@blob)
@blob.download do |chunk|
response.stream.write(chunk)
end
ensure
response.stream.close
end
end
end
end
end
# config/initializers/core_extentions.rb
Dir[File.expand_path(File.join(File.dirname(File.absolute_path(__FILE__)), 'lib/core_extensions')) + "/**/*.rb"].each do |file|
require file
end
ActiveStorage::RepresentationsController.prepend CoreExtensions::ActiveStorage::Representation::Downloader
ActiveStorage::BlobsController.prepend CoreExtensions::ActiveStorage::Blob::Downloader
# lib/core_extensions/active_storage/headers.rb
module CoreExtensions
module ActiveStorage
module Headers
extend ActiveSupport::Concern
private
def set_headers(blob)
# Hard coded this to 365, simply because thats what I want tbh
expires_in 365.days, public: true
response.headers["Content-Type"] = blob.content_type
# Commented this out because in Rails 5.2.3 this isn't a thing
# response.headers["Content-Disposition"] = ActionDispatch::Http::ContentDisposition.format(
# disposition: params[:disposition] || "inline",
# filename: blob.filename.sanitized
# )
end
end
end
end
# lib/core_extensions/active_storage/representation/downloader.rb
# had to make the file a different name because reasons ¯\_(ツ)_/¯
module CoreExtensions
module ActiveStorage
module Representation
module Downloader
include CoreExtensions::ActiveStorage::Headers
def show
return super unless params[:proxy] == 'true'
set_headers(@blob)
variant = @blob.representation(params[:variation_key]).processed
@blob.service.download(variant.key) do |chunk|
response.stream.write(chunk)
end
ensure
response.stream.close
end
end
end
end
end
@ArthurTruong5

This comment has been minimized.

Copy link

@ArthurTruong5 ArthurTruong5 commented Mar 24, 2020

I'm getting NameError: uninitialized constant CoreExtensions

@joelrojo

This comment has been minimized.

Copy link

@joelrojo joelrojo commented Mar 27, 2020

In line 2 of core_extensions.rb, I had to change the Dir path to: "#{Rails.root}/lib/core_extensions/**/*.rb"

@ArthurTruong5

This comment has been minimized.

Copy link

@ArthurTruong5 ArthurTruong5 commented Mar 28, 2020

In line 2 of core_extensions.rb, I had to change the Dir path to: "#{Rails.root}/lib/core_extensions/**/*.rb"

Could you show me your full file? Now I'm getting NameError: uninitialized constant CoreExtensions::ActiveStorage::Headers

I'm using rails 6

@joelrojo

This comment has been minimized.

Copy link

@joelrojo joelrojo commented Mar 28, 2020

I also had to put the files in app/lib rather than at the root /lib. Since Rails wasn't autoloading before boot in production, it was giving me NameError.

Still couldn't get Cloudfront to work with variants for some reason, so I ended up scrapping the monkey patch and leveraged ImageKit (referral link 🙏🏽), which is actually pretty dope. Easy to work with since it plugs directly into S3 and handles all the variant and cropping I was looking to do with ActiveRecord. I'm no longer using ActiveRecord variants.

Then in the API response, instead of using url_for(image), I created an application helper that builds a unique url for ImageKit based on the blob key, so it hits S3 directly and bypasses the server.

@timm-oh

This comment has been minimized.

Copy link
Owner Author

@timm-oh timm-oh commented May 11, 2020

Hey @ArthurTruong5,

I'm sorry for the late reply, I missed these comments in my mailbox.

I realised that in the application that I've been working on, a previous developer added this line to the application.rb and hence why I didn't run into this problem.

config.eager_load_paths += %W(#{config.root}/lib)

Did you come right with this problem or is it still throwing a NameError?

@timm-oh

This comment has been minimized.

Copy link
Owner Author

@timm-oh timm-oh commented May 11, 2020

Hey @joelrojo,

Thats a little strange that it didn't work for variants for you.
When I created this monkey patch that was the first thing I was testing with.

What was happening when you were trying to implement it with variants?
Was it still hitting your server and redirecting to s3?

@Grey-worm

This comment has been minimized.

Copy link

@Grey-worm Grey-worm commented May 15, 2020

Hi

I have tried this and I was able successfully to get this to work. Though my cloud front setup is having issues (which I'm resolving), how you plan to handle the existing objects in S3 with a new CloudFront service in place. Do we need to migrate things or something?

@timm-oh

This comment has been minimized.

Copy link
Owner Author

@timm-oh timm-oh commented May 15, 2020

hey @Grey-worm,

Could you elaborate a little more in regards with existing objects?
I'm not following here

@Grey-worm

This comment has been minimized.

Copy link

@Grey-worm Grey-worm commented May 15, 2020

Hi @howler,

Ideally ActiveStorage blob will provide us the s3 object key to search at S3 to retrieve/generate the URL. Now that we pointed the storage location to Cloudfront host configured on the bucket, Do we need to configure any thing additionally at cloudfront side(I mean any policies or OAI settings) or will it simply fetch from cloudfront.

Note: I'm new to ActiveStorage, correct me if I didn't convey any thing in right way.

@timm-oh

This comment has been minimized.

Copy link
Owner Author

@timm-oh timm-oh commented May 15, 2020

hey @Grey-worm,

If you comment out the code in core_extentions.rb and you call url_for on an active storage object (blob or variant), what is the url that you end up with?

Is it something like bucket-name.s3-region.amazonaws.com or xxxx.cloudfront.net?

@Grey-worm

This comment has been minimized.

Copy link

@Grey-worm Grey-worm commented May 16, 2020

Without core_extentions.rb snippet in place

url_for: /rails/active_storage/..... [redirects to http://bucket-name.xxx.cloudfront.net/key/...]
via proxy_url method: xxx.cloudfront.net:3000/rails/active_storage/...
object.image.service_url: bucket-name.xxx.cloudfront.net/key/...

@timm-oh

This comment has been minimized.

Copy link
Owner Author

@timm-oh timm-oh commented May 16, 2020

@Grey-worm okay cool.

I'm not really seeing the need for you to put this in place then.
This monkey patch would be used in the instance where you have CDN in front of your rails application.

@Grey-worm

This comment has been minimized.

Copy link

@Grey-worm Grey-worm commented May 19, 2020

Okay

@yhk1038

This comment has been minimized.

Copy link

@yhk1038 yhk1038 commented Jun 4, 2020

In application_helper.rb, the second parameter options of the method proxy_url should have default value {} like,

# on above
def proxy_url(active_storage_item, options)
end

# should be
def proxy_url(active_storage_item, options = {})
end

To proxy_url(attachable) work.

@timm-oh

This comment has been minimized.

Copy link
Owner Author

@timm-oh timm-oh commented Jun 4, 2020

Hey @yhk1038,
You're definitely correct, i'll make the amendment. 👍

@cyrusstoller

This comment has been minimized.

Copy link

@cyrusstoller cyrusstoller commented Feb 13, 2021

Any tips on how to handle this in rails v6.1?

I added this to my production.rb

  config.action_controller.asset_host = ENV['CLOUDFRONT']
  config.active_storage.service = :amazon
  config.active_storage.delivery_method = :proxy

And then tweaked your helper to

  def proxy_url(active_storage_item, options = {})
    options.merge!(host: ENV['CLOUDFRONT']) if ENV['CLOUDFRONT'].present?
    polymorphic_url(active_storage_item, options)
  end
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment