belgoros/AWS S3-Paperclip5-Heroku.md

## AWS S3-Paperclip5-Heroku.md

      
    Raw
  

              AWS S3-Paperclip5-Heroku.md
            
          
    Setting Amazon S3 for Paperclip and Heroku

The latest Paperclip release 5.1.0 has changed a little bit the way to
set it up with Amazon S3 service (Amazon Simple Storage Service). Moreover, after googling a lot here and there, we could see many solutions and settings,
some of them being outdated, some - often different and did not work well.
So I decided to summarize in one replace all the steps needed to set up your Rails application deployed on Heroku and be able to use it with Paperclip 5 and Amazon S3 service.
In case you don't know, Heroku does not allow your Rails application to write and offers read only access. What means that you can't use Paperclip and save your files to Heroku's file system.
So you will have to find a way to upload/store/read your files. As stated in Paperclip documentation, Paperclip ships with 3 storage adapters:

File Storage
S3 Storage (via aws-sdk)
Fog Storage

If you would like to use Paperclip with another storage, you can install these gems along side with Paperclip:

paperclip-azure
paperclip-azure-storage
paperclip-dropbox

For our case, we'll  choose S3 Storage option.
Rails application settings (to store file locally)

So the first thing first, let's go and install the required gems.

Add the following gems to your Rails application Gemfile:

paperclip: file attachment library for ActiveRecord
aws-sdk:  to store your files using Amazon's S3 service


gem 'paperclip', '~> 5.1.0'
gem 'aws-sdk',   '~> 2.6'


As required by Paperclip, ImageMagick must be installed and Paperclip must have access to it.


Run bundle in your terminal to install the above gems.


Create an account at Amazon Simple Storage Service (s3).


In my example, I would like to attach an image to a Question model. You could have different model and business requirements. See another example from Paperclip Quick start guide. So go on and add the following settings to your Rails app model:


class Question < ActiveRecord::Base
    has_attached_file :photo,
                       styles: { medium: "100x100>", thumb: "150x150>" }, default_url: "/images/:style/missing.png"
    validates_attachment :photo,
                        content_type: { content_type: ["image/jpeg", "image/gif", "image/png"] },
                        size: { in: 0..5.megabytes }
end


Create a new migration to enable Paperclip coupling with the Question model:

class AddPhotoColumnToQuestions < ActiveRecord::Migration
  def up
    add_attachment :questions, :photo
  end

  def down
    remove_attachment :questions, :photo
  end
end

The above migration will add the following columns to your corresponding model table (questions in my case):
+ photo_file_name
+ photo_content_type
+ photo_file_size
+ photo_updated_at
7. Run rake db:migrate to execute the migration.
8. Create/modify your view form to be able to attach a file to your model. Here is my example of new question form:
= simple_form_for @question do |f|
  = f.error_notification

  = f.input :text
  = f.input :photo, as: :file

  = f.button :submit

I'm using Simple Form and Slim here, but you can find a classic forms example in Paperclip views section.

Your application controller:

add the photo to required params:

def question_params
  params.require(:question).permit(:text, :photo)
  #params.require(:user).permit(:avatar) - from Paperclip docs
end


create action/method of the controller:

@question = Questions.new(question_params)
respond_to do |format|
  if @question.save
    ...
  else
    ...
  end
end


Now you should be able to save images locally. The files that are assigned as attachments are, by default, placed in the directory specified by the :path option to has_attached_file. By default, this location is :rails_root/public/system/:class/:attachment/:id_partition/:style/:filename.
Amazon S3 settings


Create a free account at Amazon's S3. No panic, it will take you some time to set up a new account from scratch. You can skip some options and set them later. After you confirmed your account  at Amazon S3, you should be abe to sign in.
To be able to store file in Amazon S3 service, you should first create an appropriate place for that. Amazon S3 calls them bucket. So go on and create a new bucket:

click on  Services menu and choose S3 in the long list of available   services.
you should be redirected to S3 [service page](https://  console.aws.amazon.com/s3/home?region=us-east-1)
click on Create Bucket blue button on the left
enter a name of the bucket in the Name, let's call  it  appdev.
choose a preferred region in the drop down Region list. You should choose  the one that is closer to your  clients location. I  chosen Franfurt.    There is also  Ireland for EU locations.
click on Create button to validate.


After the bucket created, you will see it in the list of your available buckets page on the left and the created bucket settings on the right. If you have several buckets in the list, you can always display their settings by selecting/highlighting  a bucket and click the Properties button in the left section button. You can also delete a bucket via Actions drop-down menu in the left side section of the page and access the bucket properties via Properties menu from that drop-down menu as well.

Amazon Identity and Access Management (IAM)

Now we have to create a user to access the created earlier bucket and assign him permissions.

Suppose that you signed in to Amazon console. Click on Services drop-list menu and select IAM service in the list. You should be able to see the IAM page.
Click on Users menu in the left-side menu to see existing users. We are going to create a new one. Go on and click on  Create New Users blue button.
You can create 5 users at once. In our case we need just the only one. So enter a name for the user to create, par ex. tester. Leave the checkbox Generate an access key for each user checked and click on Create.
On the displayed page you should see a successful creation message and be able to display security credentials of the user(s). For that, just click on  Show User Security Credentials link. You can also download the credentials in CSV format by clicking on Download Credentials button in the bootom-right side of the page.
The credentials are generated as a hash with Access Key ID value and Secret Access Key keys. We are  going to use them later in our Rails configuration files.
Click on Close button to come  back to the users page.
Click  on our tester user in the  list to see his properties.
You will see the  4 tabs: Groups, Permissions, Security Credentials and Access Advisor.
Click on Permissions tab.
Click on Attach Policy button to attach a  security policy to our user. Amazon offers a list of pre-configured policies, but you can create your own, it is just a JSON configuartion file with specified key/values pairs.
In the displayed policies list either find AmazonS3FullAccess policy or filter to find it by entering just S3 value in Policy Type text field on the top of the policies table.
Check the checkbox  of the AmazonS3FullAccess policy and click on Attach Policy button in the bottom of the page.
You can delete a  User, detach  his policy, etc. at any moment.
Note the value of  User ARN field that looks like that:
arn:aws:iam::531460575238:user/tester you will need it later. User ID value 531460575238 and your user name  will be different.

Put together User and Bucket policies


Return to S3 service by selecting it in  Services drop-list menu. Amazon keeps the history of frequently used services so that you will not have to search them again and again. So you will have just to click on S3 service icon in History section on the left side of the page.
Click on the bucket appdev we created earlier. As we  haven't yet uploaded any files to the bucket, it is empty.
Click on Properties button on the top-right. You will see all your bucket settings (its name, region, creation date, etc) as well as  other sections (permissions, static web site hosting, etc.).
Click on Permissions section of the bucket. You will see the default permission attached  when the bucket was created. We are going to attache a new policy to the bucket. For that:

click on Add bucket policy  button;
you can either copy-paste an existing policy JSON or use  AWS Policy generator. So go on and click on AWS Policy generator link in the bottom of the dialog window;
in the new opened navigator tab, you will have to set up values for your future policy:

choose S3 Bucket Policy in  Select Type of Policy drop-down list;
leave Effect value on Allow selected in the  Step 2 Add Statement(s) section;
copy/paste the value of Principal  (see p. 14 in section Amazon Identity and Access Management (IAM) regarding User ARN value);
select Amazon S3 value in  AWS Service drop-down list
check All actions checkbox on the left of Actions text field;
enter a  value corresponding your bucket according to the specified format indicated under the text field Amazon Resource Name (ARN). The mine looks like that: arn:aws:s3:::appdev.
click on  Add statement button that should be enabled once you entered all the required values.
if everything is correct, you will see the message

You added the following statements. Click the button below to Generate a policy.

and some short settings of  the newly generated statement.

click  Generate Policy button.
you will  be presented a generated Policy JSON Doculment in a dialog popup window that looks like that:


{
  "Id": "Policy1477918632090",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1477918493899",
      "Action": "s3:*",
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::appdev",
      "Principal": {
        "AWS": [
          "arn:aws:iam::531460575238:user/tester"
        ]
      }
    }
  ]
}

    + copy the displayed JSON content, return to the tab where you came from (see p.4 above about **AWS Policy generator**) and paste the copied JSON.
    + click **Save** and **Close** link to close the dialog window.


You can always check/edit the  bucket policy via the button Properies available when selecting a bucket from the list. When clicking on Permissions section link, now you will see the option Edit bucket policy. if you click on the link, you should see the policy JSON we generated earlier.

That is all for AWS S3 and IAM settings.
Add AWS S3 settings to your Rails application

We'll have to add some environnement variables and set them on Heroku to connect our Rails app to AWS S3 via Heroku with Paperclip.
I'm using Figaro gem to manage these variables, but you are free to choode whatever you want, all of them work the same way.

I suppose that you already have your application configuration management gem installed and you can call ENV in your environment files.
Add the following settings to your config/environment/production.rb file:

Rails.application.configure do
...
    config.paperclip_defaults = {
        storage: :s3,
        s3_credentials: {
          bucket: ENV['S3_BUCKET_NAME'],
          access_key_id: ENV['AWS_ACCESS_KEY_ID'],
          secret_access_key: ENV['AWS_SECRET_ACCESS_KEY'],
          s3_region: ENV['S3_REGION'],
          s3_host_name: ENV['S3_HOST_NAME']
        }
    }
...
end

Take care: there a new attribute: s3_region needed if you use Paperclip 5.

Add the following keys to your application.yml file:

S3_BUCKET_NAME:
AWS_ACCESS_KEY_ID:
AWS_SECRET_ACCESS_KEY:
S3_REGION:
S3_HOST_NAME:

If you use an other gem, proceed as required in their documentation to add key/value pairs.
 #### Values examples:
 S3_BUCKET_NAME: appdev
 AWS_ACCESS_KEY_ID: YOUR_ACCESS_KEY_FROM_AMAZON
 AWS_SECRET_ACCESS_KEY: YOUR_SECRET_KEY_FROM_AMAZON
 S3_REGION: eu-central-1
 S3_HOST_NAME: s3.eu-central-1.amazonaws.com

You can set these values specific to your environments (dev, test, staging, prod) by putting the above paperclip_defaults settings in the corresponding environment file (test.rb, development.rb, etc.)
4. These settings set up, now we are going to upload them to Heroku. You have 2 ways of doing that: either via GUI of your Rails application provided on your Heroku account application space, or from the command line as follows:
 heroku config:set YOUR_ENV_KEY=your_key_value

for example:
 heroku config:set S3_BUCKET_NAME=appdev

Using the figaro command, you can set values from your configuration file all at once:
figaro heroku:set -e production


Restart your Heroku app with heroku restart and you should be done.

When checking if everything works, take a look in your browser Dev Tools (console) as well a in your Heroku logs (heroku logs) to see that something goes as needed. You can see, for example, if an image was downloaded/uploaded to amazon s3 service, or see errors log from Amazon as well. It will make it possible to find the reason.
Voilà. Hope this helps.