Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Simple walkthrough for adding direct (S3) uploads to a Roda & Sequel app with Shrine

Adding Direct Uploads to a Roda & Sequel App with Shrine

Installation

Add Shrine to the Gemfile:

# Gemfile

gem "shrine", "~> 2.9"

Configuration

Create an initializer that will be loaded when your app boots, where you configure your storage and load initial plugins.

# config/shrine.rb

require "shrine"
require "shrine/storage/file_system"

Shrine.storages = {
  cache: Shrine::Storage::FileSystem.new("public", prefix: "uploads/cache"),
  store: Shrine::Storage::FileSystem.new("public", prefix: "uploads/store"),
}

Shrine.plugin :sequel # load integration for the Sequel ORM
Shrine.plugin :cached_attachment_data # for forms

Uploader

Create an uploader for the types of files you'll be uploading:

# uploaders/image_uploader.rb

class ImageUploader < Shrine
end

Now add an attachment attribute to your model:

# models/article.rb

class Article < Sequel::Model
  include ImageUploader::Attachment.new(:cover_photo)
end

You'll also need to add the <attachment>_data text or JSON column to that table:

Sequel.migration do
  change do
    add_column :articles, :cover_photo_data, :text
  end
end

View

In your model form you can now add form fields for the attachment attribute, and an image tag for the preview:

<input type="hidden" name="article[cover_photo]" value="<%= @article.cached_cover_photo_data %>" class="upload-hidden">
<input type="file" name="article[cover_photo]" class="upload-file">
<img class="upload-preview">

The file field will be used for choosing files, and the hidden field for storing uploaded file data and retaining it across form redisplays in case of validation errors.

You should now be able to upload the images via the form, and display them in your views:

<img src="<%= @article.cover_photo_url %>" width=500>

Direct upload

We can now add asynchronous direct uploads to the mix. We'll be using a JavaScript file upload library called Uppy, along with Shrine's upload_endpoint plugin.

Upload endpoint

On the server side we'll need to add an endpoint that accept uploads that Uppy forwards. Shrine's upload_endpoint plugin gives you a complete Rack endpoint that forwards received files to the specified storage. All we need to do is load the plugin and mount the endpoint to the desired path.

# config/shrine.rb

# ...
Shrine.plugin :upload_endpoint
# app.rb

class App < Roda
  route do |r|
    r.on "images/upload" do
      r.run ImageUploader.upload_endpoint(:cache)
    end

    # ...
  end
end

Uppy

Now we can setup Uppy to do the direct uploads. The easiest way to install it is to pull the JavaScript and CSS files from unpkg:

<!DOCTYPE html>
<html>
  <head>
    <!-- polyfill needed for Uppy and our custom script -->
    <script src="https://unpkg.com/babel-polyfill@6.26.0/dist/polyfill.min.js"></script>
    <!-- Uppy script -->
    <script src="https://unpkg.com/uppy@0.22.0/dist/uppy.min.js"></script>
    <!-- Uppy stylesheet -->
    <link href="https://unpkg.com/uppy@0.22.0/dist/uppy.min.css" rel="stylesheet" />
  </head>

  <body>
    ...
  </body>
</html>

Now we can add the following JavaScript code which will perform direct uploads to Shrine's upload endpoint when the user selects the file, assigning the results to the hidden attachment field to be submitted:

function fileUpload(fileInput) {
  var imagePreview = fileInput.parentNode.querySelector('.image-preview')

  fileInput.style.display = 'none' // uppy will add its own file input

  var uppy = Uppy.Core({
      id:                  fileInput.id,
      thumbnailGeneration: false,
    })
    .use(Uppy.FileInput, {
      target:             fileInput.parentNode,
      allowMultipleFiles: false,
    })
    .use(Uppy.Informer, {
      target: fileInput.parentNode,
    })
    .use(Uppy.ProgressBar, {
      target: imagePreview.parentNode,
    })

  uppy.use(Uppy.XHRUpload, {
    endpoint: '/images/upload', // Shrine's upload endpoint
    fieldName: 'file',
    headers: { 'X-CSRF-Token': document.querySelector('meta[name=_csrf]').content }
  })

  uppy.run()

  uppy.on('upload-success', function (fileId, data) {
    // retrieve uppy's file object (`file.data` contains the actual JavaScript File object)
    var file = uppy.getFile(fileId)

    // show image preview
    imagePreview.src = file.preview

    // read uploaded file data from the upload endpoint response
    var uploadedFileData = JSON.stringify(data)

    // set hidden field value to the uploaded file data so that it's submitted with the form as the attachment
    var hiddenInput = fileInput.parentNode.querySelector('.upload-hidden')
    hiddenInput.value = uploadedFileData
  })

  return uppy
}

document.querySelectorAll('input[type=file]').forEach(function (fileInput) {
  fileUpload(fileInput)
})

And that's it, now when a file is selected it will be asynchronously uploaded to your app. During the upload a nice progress bar will be displayed, and when the upload finishes an image preview will be shown.

Adding Direct S3 Uploads to a Roda & Sequel App with Shrine

AWS S3 setup

You'll need to create an AWS S3 bucket, which is where the uploads will be stored. See this walkthrough on how to do that.

Next you'll need to configure CORS for that bucket, so that it accepts uploads directly from the browser. To add the simplest (and the most permissible) CORS configuration, install the aws-sdk-s3 gem and run this script:

require "aws-sdk-s3"

client = Aws::S3::Client.new(
  access_key_id:     "<YOUR KEY>",
  secret_access_key: "<YOUR SECRET>",
  region:            "<REGION>",
)

client.put_bucket_cors(
  bucket: "<YOUR BUCKET>",
  cors_configuration: {
    cors_rules: [{
      allowed_headers: ["Authorization", "Content-Type", "Origin"],
      allowed_methods: ["GET", "POST"],
      allowed_origins: ["*"],
      max_age_seconds: 3000,
    }]
  }
)

Installation

Add Shrine and aws-sdk-s3 to the Gemfile:

# Gemfile

gem "shrine", "~> 2.9"
gem "aws-sdk-s3", "~> 1.2"

Configuration

Create an initializer that will be loaded when your app boots, and replace placeholders with the actual credentials of your S3 bucket.

# config/shrine.rb

require "shrine"
require "shrine/storage/s3"

s3_options = {
  access_key_id:     "<YOUR_ACCESS_KEY_ID>",
  secret_access_key: "<YOUR_SECRET_ACCESS_KEY>",
  bucket:            "<YOUR_BUCKET>",
  region:            "<YOUR_REGION>",
}

Shrine.storages = {
  cache: Shrine::Storage::S3.new(prefix: "cache", **s3_options),
  store: Shrine::Storage::S3.new(prefix: "store", **s3_options),
}

Shrine.plugin :sequel # load integration for the Sequel ORM
Shrine.plugin :cached_attachment_data # for forms

Uploader

Create an uploader for the types of files you'll be uploading:

# uploaders/image_uploader.rb

class ImageUploader < Shrine
end

Now add an attachment attribute to your model:

# models/article.rb

class Article < Sequel::Model
  include ImageUploader::Attachment.new(:cover_photo)
end

You'll also need to add the <attachment>_data text or JSON column to that table:

Sequel.migration do
  change do
    add_column :articles, :cover_photo_data, :text
  end
end

View

In your model form you can now add form fields for the attachment attribute, and an image tag for the preview:

<input type="hidden" name="article[cover_photo]" value="<%= @article.cached_cover_photo_data %>" class="upload-hidden">
<input type="file" name="article[cover_photo]" class="upload-file">
<img class="upload-preview">

The file field will be used for choosing files, and the hidden field for storing uploaded file data and retaining it across form redisplays in case of validation errors.

You should now be able to upload the images via the form, and display them in your views:

<img src="<%= @article.cover_photo_url %>" width=500>

Direct upload

We can now add asynchronous direct uploads to the mix. We'll be using a JavaScript file upload library called Uppy.

Direct file uploads to S3 from the browser work in the following way:

  1. User selects the file
  2. On client side we fetch request params and URL for the S3 upload from the app
  3. Using this information we upload the file to S3

Presign endpoint

So, on the server side we'll need to add an endpoint which returns valid request params and URL for the S3 upload. Shrine's presign_endpoint plugin bakes this functionality in, all we need to do is load the plugin and mount the endpoint to the desired path:

# config/shrine.rb

# ...
Shrine.plugin :presign_endpoint
# app.rb

class App < Roda
  route do |r|
    r.on "presign" do
      r.run Shrine.presign_endpoint(:cache)
    end

    # ...
  end
end

Uppy

Now we can setup Uppy to do the direct uploads. The easiest way to install it is to pull the JavaScript and CSS files from unpkg:

<!DOCTYPE html>
<html>
  <head>
    <!-- polyfill needed for Uppy and our custom script -->
    <script src="https://unpkg.com/babel-polyfill@6.26.0/dist/polyfill.min.js"></script>
    <!-- polyfill for window.fetch -->
    <script src="https://unpkg.com/whatwg-fetch@2.0.3/fetch.js"></script>
    <!-- Uppy script -->
    <script src="https://unpkg.com/uppy@0.22.0/dist/uppy.min.js"></script>
    <!-- Uppy stylesheet -->
    <link href="https://unpkg.com/uppy@0.22.0/dist/uppy.min.css" rel="stylesheet" />
  </head>

  <body>
    ...
  </body>
</html>

Now we can add the following JavaScript code which will perform direct uploads to S3 when the user selects the file, using Shrine's presign endpoint, assigning the results to the hidden attachment field to be submitted:

function fileUpload(fileInput) {
  var imagePreview = fileInput.parentNode.querySelector('.image-preview')

  fileInput.style.display = 'none' // uppy will add its own file input

  var uppy = Uppy.Core({
      id:                  fileInput.id,
      thumbnailGeneration: false,
    })
    .use(Uppy.FileInput, {
      target:             fileInput.parentNode,
      allowMultipleFiles: false,
    })
    .use(Uppy.Informer, {
      target: fileInput.parentNode,
    })
    .use(Uppy.ProgressBar, {
      target: imagePreview.parentNode,
    })

  uppy.use(Uppy.AwsS3, {
    getUploadParameters: function (file) {
      return fetch('/presign?filename=' + file.name) // Shrine's presign endpoint
        .then(function (response) { return response.json() })
    }
  })

  uppy.run()

  uppy.on('upload-success', function (fileId, data) {
    // retrieve uppy's file object (`file.data` contains the actual JavaScript File object)
    var file = uppy.getFile(fileId)

    // show image preview
    imagePreview.src = file.preview

    // construct uploaded file data in the format that Shrine expects
    var uploadedFileData = JSON.stringify({
      id: file.meta['key'].match(/^cache\/(.+)/)[1], // remove the Shrine storage prefix
      storage: 'cache',
      metadata: {
        size:      file.size,
        filename:  file.name,
        mime_type: file.type,
      }
    })

    // set hidden field value to the uploaded file data so that it's submitted with the form as the attachment
    var hiddenInput = fileInput.parentNode.querySelector('.upload-hidden')
    hiddenInput.value = uploadedFileData
  })

  return uppy
}

document.querySelectorAll('input[type=file]').forEach(function (fileInput) {
  fileUpload(fileInput)
})

And that's it, now when a file is selected it will be asynchronously uploaded directly to your S3 bucket. During the upload a nice progress bar will be displayed, and when the upload finishes an image preview will be shown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment