Speed up the AWS Elastic Beanstalk of your Rails deployment with bundle/assets caching

cache.yml

# .ebextensions/cache.config
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/02a_set_cache.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/bash
      set -xe

      EB_APP_STAGING_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)

      BUCKET="elasticbeanstalk-PUT_YOUR_REGION-PUT_YOUR_ACCOUNT_ID"
      mkdir -p /tmp/cache
      if aws s3 ls s3://${BUCKET}/cache/gem.tar.gz ; then
        aws s3 cp --quiet s3://${BUCKET}/cache/gem.tar.gz /tmp/cache/gem.tar.gz
      fi
      if aws s3 ls s3://${BUCKET}/cache/node_modules.tar.gz ; then
        aws s3 cp --quiet s3://${BUCKET}/cache/node_modules.tar.gz /tmp/cache/node_modules.tar.gz
      fi
      if aws s3 ls s3://${BUCKET}/cache/assets.tar.gz ; then
        aws s3 cp --quiet s3://${BUCKET}/cache/assets.tar.gz /tmp/cache/assets.tar.gz
      fi

      [ -f /tmp/cache/gem.tar.gz ]          && tar -xf /tmp/cache/gem.tar.gz          -C $EB_APP_STAGING_DIR && chown webapp:webapp -R "$EB_APP_STAGING_DIR/vendor/bundle"
      [ -f /tmp/cache/node_modules.tar.gz ] && tar -xf /tmp/cache/node_modules.tar.gz -C $EB_APP_STAGING_DIR && chown webapp:webapp -R "$EB_APP_STAGING_DIR/node_modules"
      [ -f /tmp/cache/assets.tar.gz ]       && tar -xf /tmp/cache/assets.tar.gz       -C $EB_APP_STAGING_DIR && chown webapp:webapp -R "$EB_APP_STAGING_DIR/tmp"
      exit 0

  "/opt/elasticbeanstalk/hooks/appdeploy/pre/11a_build_cache.sh":
    mode: "000755"
    owner: root
    group: root
    content: |
      #!/bin/bash
      set -xe

      EB_APP_STAGING_DIR=$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_dir)

      cd $EB_APP_STAGING_DIR
      mkdir -p /tmp/cache
      tar -zcf /tmp/cache/gem.tar.gz          vendor/bundle
      tar -zcf /tmp/cache/node_modules.tar.gz node_modules
      tar -zcf /tmp/cache/assets.tar.gz       tmp/cache/assets

      BUCKET="elasticbeanstalk-PUT_YOUR_REGION-PUT_YOUR_ACCOUNT_ID"
      aws s3 cp /tmp/cache/gem.tar.gz          s3://${BUCKET}/cache/gem.tar.gz
      aws s3 cp /tmp/cache/node_modules.tar.gz s3://${BUCKET}/cache/node_modules.tar.gz
      aws s3 cp /tmp/cache/assets.tar.gz       s3://${BUCKET}/cache/assets.tar.gz

Thanks, gave me the right ideas, though I had to extend it a little bit to speed up also launching new instances. My trick is to put the gzipped files onto the s3 bucket so new instances can use those.

@ssaunier Thanks a lot for that! It has actually improved ten fold the deploy time specially when assets pre-compilation for a given project is too expensive. One consideration is that somehow the current rails stack will use root user and vendor/bundle is not available and will fail the tar command.

I'd appreciate if you can share the other parts of your stack scripts mentioned on the blog post. Specially the one about sidekiq.

Keep Rocking!

@TomK32, could you post your script involving S3 instead of /tmp storage? Thanks!

I updated my gist with commands to push the cache to the Elastic Beanstalk S3 bucket. You'll find it in your console

One consideration is that somehow the current rails stack will use root user and vendor/bundle is not available and will fail the tar command.

Is there a fix for that?

Thanks for posting this. If I want to use this with multiple apps all running in the same region, does it stand to reason that modifying the BUCKET would be the right way to support that?

e.g.

BUCKET="elasticbeanstalk-PUT_YOUR_REGION-PUT_YOUR_ACCOUNT_ID-APP_NAME"

Also, I assume I can reference my app's env vars during 02a_set_cache.sh?

Thanks again!