Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
AWS Elastic Beanstalk Ruby 2.0/Puma Environment - .ebextensions tweaks and Sidekiq configuration. This is known to work fine with AWS Elastic Beanstalk 's 64bit Amazon Linux 2014.03 v1.0.1 running Ruby 2.0 (Puma) stack. Later stack versions might not work, but for that specific version works fine.
# Install Git needed for Git based gems
packages:
yum:
git: []
# Fixing permissions of packaged gems
files:
"/opt/elasticbeanstalk/hooks/appdeploy/pre/10_fixing_permission.sh":
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/containerfiles/envvars
CACHE_GEM_DIR=$EB_CONFIG_APP_ONDECK/vendor/cache
if [ -d $CACHE_GEM_DIR ]
then
chown -R webapp:webapp $CACHE_GEM_DIR
echo "Modified the owner of $CACHE_GEM_DIR files"
else
echo "Nothing in $CACHE_GEM_DIR"
fi
true
mode: "000755"
# Modified system bundle script to run 'bundle package'
files:
"/opt/elasticbeanstalk/hooks/appdeploy/pre/10_bundle_install.sh":
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/containerfiles/envvars
cd $EB_CONFIG_APP_ONDECK
if [ -f Gemfile ]
then
echo "running 'bundle install' with Gemfile:"
cat Gemfile
if [ -d $EB_CONFIG_APP_ONDECK/vendor/cache ]
then
/usr/local/bin/bundle install --local
# Incase there is a gem that is missing from the cache
/usr/local/bin/bundle pack --all
/usr/local/bin/bundle install
else
/usr/local/bin/bundle pack --all
/usr/local/bin/bundle install
fi
if [ $? != 0 ]
then
echo "ERROR: bundle install failed!"
exit 1
else
echo "bundle install succeeded"
fi
else
echo "no Gemfile found! Skipping bundle install stage!"
fi
if [ -f Gemfile.lock ]
then
echo "encountered a Gemfile.lock, setting proper permissions"
chown $EB_CONFIG_APP_USER:$EB_CONFIG_APP_USER Gemfile.lock
else
echo "no Gemfile.lock file found, so no permissions to set on it";
fi
true
mode: "000755"
# Sidekiq interaction and startup script
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq":
mode: "000755"
content: |
#!/bin/bash
. /opt/elasticbeanstalk/containerfiles/envvars
PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq.pid
cd $EB_CONFIG_APP_CURRENT
if [ -f $PIDFILE ]
then
SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
if [ -z $SIDEKIQ_LIVES ]
then
rm -rf $PIDFILE
else
kill -TERM `cat $PIDFILE`
sleep 10
rm -rf $PIDFILE
fi
fi
BUNDLE=/usr/local/bin/bundle
SIDEKIQ=/usr/local/bin/sidekiq
$BUNDLE exec $SIDEKIQ \
-e production \
-P /var/app/containerfiles/pids/sidekiq.pid \
-C /var/app/current/config/sidekiq.yml \
-L /var/app/containerfiles/logs/sidekiq.log \
-d
"/opt/elasticbeanstalk/hooks/appdeploy/pre/03_mute_sidekiq":
mode: "000755"
content: |
#!/bin/bash
. /opt/elasticbeanstalk/containerfiles/envvars
PIDFILE=$EB_CONFIG_APP_PIDS/sidekiq.pid
if [ -f $PIDFILE ]
then
SIDEKIQ_LIVES=$(/bin/ps -o pid= -p `cat $PIDFILE`)
if [ -z $SIDEKIQ_LIVES ]
then
rm -rf $PIDFILE
else
kill -USR1 `cat $PIDFILE`
sleep 10
fi
fi
# Globally include the vendor/cache gem location and the location of puma_http11 gem
files:
"/opt/elasticbeanstalk/containerfiles/envvars.d/appenv":
content: |
export RUBYLIB=$RUBYLIB:/var/app/current/vendor/cache:/usr/local/lib64/ruby/gems/2.0/gems/puma-2.8.1/lib/
mode: "000644"

thanks for this. could you add your /opt/elasticbeanstalk/containerfiles/envvars file? This is throwing errors for me (specifically the sidekiq.config) without it

Dumb question, was this deployed as a worker or webapp EB environment?

Hi guys, please note make sure you use the 64bit Amazon Linux 2014.03 v1.0.1 running Ruby 2.0 (Puma) stack (Web Server Tier), because the later released stack (1.0.2 and 1.0.3) broke my script. So I suggest using 1.0.1 if you want to test my .ebextensions

Check to see if changing "post" to "enact" may help?

I actually got it to work with a few modifications on the 64bit Amazon Linux 2014.03 v1.0.2 running Ruby 2.0 (Passenger Standalone) stack. The most notable change was changing
. /opt/elasticbeanstalk/containerfiles/envvars to . /opt/elasticbeanstalk/support/envvars

Here is everything in my .ebextensions folder

It's working great for me. No ssh'ing at any point

t2 commented Feb 11, 2015

There must have been some changes since these were created. 64bit Amazon Linux 2014.09 v1.1.0 running Ruby 2.1 (Puma) will not play nice. Anyone had luck recently?

@t2 from your own gist https://gist.github.com/t2/c629e1018a0f6815d871 :P
works for me on 1.2.0

jolks commented Mar 3, 2015

This with @t2 gist work perfectly for Configuration 64bit Amazon Linux 2014.09 v1.2.0 running Ruby 2.1 (Passenger Standalone). What I am using can be found here. Thank you everyone!

t2 commented Mar 3, 2015

Yeah, did some digging and got it worked out.

Thank you @jolks

jolks commented Mar 20, 2015

Cheers @musgravejw

@gcarrion-gfrmedia Thanks for this! I am using Ruby 2.2 Puma and have made some updates:

https://gist.github.com/kpheasey/d010608c2971408bf02c

dwilkie commented Jun 27, 2015

https://github.com/mperham/sidekiq/wiki/Deployment recommends using a process supervisor rather than running sidekiq as a daemon, so it's automatically restarted if it's crashed. I followed this blog post http://qiita.com/sawanoboly/items/d28a05d3445901cf1b25 (it's in Japanese) and it works perfectly. I highly recommend this approach.

hobakill commented Aug 5, 2015

@jolks -- my precompiles are still failing.

ERROR: [Instance: i-xxxxxx] Command failed on instance. Return code: 127 Output: (TRUNCATED).../ruby/lib/ruby/gems/2.1.0/gems/bundler-1.7.6/lib/bundler/runtime.rb:222: warning: Insecure world writable dir /opt/elasticbeanstalk/lib/ruby/bin in PATH, mode 041777
bundler: command not found: sidekiq
Install missing gem executables with `bundle install`.
Hook /opt/elasticbeanstalk/hooks/appdeploy/post/50_restart_sidekiq failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.

update - never mind. working swell! thanks a ton to everyone.

Are you guys running it on the same web instances or are you using the worker tier of elastic beanstalk?

jolks commented Sep 3, 2015

@hobakill -- Sorry, i just saw your comment. Glad it worked out in the end!

jolks commented Sep 3, 2015

@Ricardonacif -- Same web instances. You just need to point your web tier to a Redis instance, in my case, ElasticCache.

@jolks -- To be clear, does that mean there is no distinguishing between your worker and web instances? Have you had any performance issues affect the end user from job load?

I'd like to have Sidekiq running on another instance then the one that does the web app. Is there any way how to do that?

TPei commented May 4, 2016

Well @nerdyglasses, it should definitely be possible. Apart from the general obstacle of aws setup it should actually be similar to how you would set it up on "traditional" server environments... The added benefit is that you have a separate Redis service to connect to anyways, so it's not really any additional work to connect from completely separate sidekiq instance...

You could start reading up on the matter with this stackoverflow post and with this blog post.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment