AWS Elastic Beanstalk Ruby 2.0/Puma Environment - .ebextensions tweaks and Sidekiq configuration. This is known to work fine with AWS Elastic Beanstalk 's 64bit Amazon Linux 2014.03 v1.0.1 running Ruby 2.0 (Puma) stack. Later stack versions might not work, but for that specific version works fine.

0000_packages.config

# Install Git needed for Git based gems
packages:
  yum:
    git: []
    

0001_fix_permissions.config

# Fixing permissions of packaged gems
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/10_fixing_permission.sh":
    content: |
        #!/usr/bin/env bash

        . /opt/elasticbeanstalk/containerfiles/envvars

        CACHE_GEM_DIR=$EB_CONFIG_APP_ONDECK/vendor/cache

        if [ -d $CACHE_GEM_DIR ]
        then
          chown -R webapp:webapp $CACHE_GEM_DIR
          echo "Modified the owner of $CACHE_GEM_DIR files"
        else
          echo "Nothing in $CACHE_GEM_DIR"
        fi

        true
    mode: "000755"

0002_bundle_pack.config

# Modified system bundle script to run 'bundle package'
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/pre/10_bundle_install.sh":
    content: |
        #!/usr/bin/env bash

        . /opt/elasticbeanstalk/containerfiles/envvars

        cd $EB_CONFIG_APP_ONDECK

        if [ -f Gemfile ]
        then
          echo "running 'bundle install' with Gemfile:"
          cat Gemfile

          if [ -d $EB_CONFIG_APP_ONDECK/vendor/cache ] 
          then
            /usr/local/bin/bundle install --local
            # Incase there is a gem that is missing from the cache
            /usr/local/bin/bundle pack --all
            /usr/local/bin/bundle install
          else 
            /usr/local/bin/bundle pack --all
            /usr/local/bin/bundle install
          fi 

          if [ $? != 0 ]
          then
            echo "ERROR: bundle install failed!"
            exit 1
          else
            echo "bundle install succeeded"
          fi

        else
          echo "no Gemfile found! Skipping bundle install stage!"
        fi

        if [ -f Gemfile.lock ]
        then
          echo "encountered a Gemfile.lock, setting proper permissions"
          chown $EB_CONFIG_APP_USER:$EB_CONFIG_APP_USER Gemfile.lock
        else
          echo "no Gemfile.lock file found, so no permissions to set on it";
        fi

        true
    mode: "000755"

0003_sidekiq.config

# Sidekiq interaction and startup script
files:
  "/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq":
    mode: "000755"
    content: |
      #!/bin/bash

      . /opt/elasticbeanstalk/containerfiles/envvars

      PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq.pid

      cd $EB_CONFIG_APP_CURRENT

      if [ -f $PIDFILE ]
      then
        SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
          if [ -z $SIDEKIQ_LIVES ]
          then
             rm -rf $PIDFILE
          else
             kill -TERM `cat $PIDFILE`
             sleep 10
             rm -rf $PIDFILE
          fi
      fi

      BUNDLE=/usr/local/bin/bundle
      SIDEKIQ=/usr/local/bin/sidekiq

      $BUNDLE exec $SIDEKIQ \
        -e production \
        -P /var/app/containerfiles/pids/sidekiq.pid \
        -C /var/app/current/config/sidekiq.yml \
        -L /var/app/containerfiles/logs/sidekiq.log \
        -d

  "/opt/elasticbeanstalk/hooks/appdeploy/pre/03_mute_sidekiq":
    mode: "000755"
    content: |
      #!/bin/bash

      . /opt/elasticbeanstalk/containerfiles/envvars

      PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq.pid

      if [ -f $PIDFILE ]
      then
        SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
          if [ -z $SIDEKIQ_LIVES ]
          then
             rm -rf $PIDFILE
          else
             kill -USR1 `cat $PIDFILE`
             sleep 10
          fi
      fi

0004_rubylibs.config

# Globally include the vendor/cache gem location and the location of puma_http11 gem
files:
  "/opt/elasticbeanstalk/containerfiles/envvars.d/appenv":
    content: |
     export RUBYLIB=$RUBYLIB:/var/app/current/vendor/cache:/usr/local/lib64/ruby/gems/2.0/gems/puma-2.8.1/lib/
    mode: "000644"

thanks for this. could you add your /opt/elasticbeanstalk/containerfiles/envvars file? This is throwing errors for me (specifically the sidekiq.config) without it

Dumb question, was this deployed as a worker or webapp EB environment?

Hi guys, please note make sure you use the 64bit Amazon Linux 2014.03 v1.0.1 running Ruby 2.0 (Puma) stack (Web Server Tier), because the later released stack (1.0.2 and 1.0.3) broke my script. So I suggest using 1.0.1 if you want to test my .ebextensions

Check to see if changing "post" to "enact" may help?

I actually got it to work with a few modifications on the 64bit Amazon Linux 2014.03 v1.0.2 running Ruby 2.0 (Passenger Standalone) stack. The most notable change was changing
. /opt/elasticbeanstalk/containerfiles/envvars to . /opt/elasticbeanstalk/support/envvars

Here is everything in my .ebextensions folder

It's working great for me. No ssh'ing at any point

This work perfectly fine on v1.0.4:
https://gist.github.com/deependersingla/c23483fc11c7f3ed4478

There must have been some changes since these were created. 64bit Amazon Linux 2014.09 v1.1.0 running Ruby 2.1 (Puma) will not play nice. Anyone had luck recently?

@t2 from your own gist https://gist.github.com/t2/c629e1018a0f6815d871 :P
works for me on 1.2.0

This with @t2 gist work perfectly for Configuration 64bit Amazon Linux 2014.09 v1.2.0 running Ruby 2.1 (Passenger Standalone). What I am using can be found here. Thank you everyone!

Yeah, did some digging and got it worked out.

Thank you @jolks

Cheers @musgravejw

@gcarrion-gfrmedia Thanks for this! I am using Ruby 2.2 Puma and have made some updates:

https://gist.github.com/kpheasey/d010608c2971408bf02c

https://github.com/mperham/sidekiq/wiki/Deployment recommends using a process supervisor rather than running sidekiq as a daemon, so it's automatically restarted if it's crashed. I followed this blog post http://qiita.com/sawanoboly/items/d28a05d3445901cf1b25 (it's in Japanese) and it works perfectly. I highly recommend this approach.

@jolks -- my precompiles are still failing.

ERROR: [Instance: i-xxxxxx] Command failed on instance. Return code: 127 Output: (TRUNCATED).../ruby/lib/ruby/gems/2.1.0/gems/bundler-1.7.6/lib/bundler/runtime.rb:222: warning: Insecure world writable dir /opt/elasticbeanstalk/lib/ruby/bin in PATH, mode 041777
bundler: command not found: sidekiq
Install missing gem executables with `bundle install`.
Hook /opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.

update - never mind. working swell! thanks a ton to everyone.

Are you guys running it on the same web instances or are you using the worker tier of elastic beanstalk?

@hobakill -- Sorry, i just saw your comment. Glad it worked out in the end!

@Ricardonacif -- Same web instances. You just need to point your web tier to a Redis instance, in my case, ElasticCache.

@jolks -- To be clear, does that mean there is no distinguishing between your worker and web instances? Have you had any performance issues affect the end user from job load?

I'd like to have Sidekiq running on another instance then the one that does the web app. Is there any way how to do that?

Well @NerdyGlasses, it should definitely be possible. Apart from the general obstacle of aws setup it should actually be similar to how you would set it up on "traditional" server environments... The added benefit is that you have a separate Redis service to connect to anyways, so it's not really any additional work to connect from completely separate sidekiq instance...

You could start reading up on the matter with this stackoverflow post and with this blog post.