Unicorn / Monit setup

unicorn

#!/bin/sh
set -e
# Example init script, this can be used with nginx, too,
# since nginx and unicorn accept the same signals

# Feel free to change any of the following variables for your app:
TIMEOUT=${TIMEOUT-60}
APP_ROOT=/path/to/your/app/current
PID=$APP_ROOT/tmp/pids/unicorn.pid
ENVIRONMENT=production
CMD="cd $APP_ROOT; bundle exec unicorn -E $ENVIRONMENT -D -c $APP_ROOT/config/unicorn.rb"
action="$1"
set -u

old_pid="$PID.oldbin"

cd $APP_ROOT || exit 1

sig () {
  test -s "$PID" && kill -$1 `cat $PID`
}

oldsig () {
  test -s $old_pid && kill -$1 `cat $old_pid`
}

workersig () {
  workerpid="$APP_ROOT/tmp/pids/unicorn.$2.pid"
  test -s "$workerpid" && kill -$1 `cat $workerpid`
}

case $action in
m start)
  sig 0 && echo >&2 "Already running" && exit 0
  su - rails -c "$CMD"
  ;;
stop)
  sig QUIT && exit 0
  echo >&2 "Not running"
  ;;
force-stop)
  sig TERM && exit 0
  echo >&2 "Not running"
  ;;
restart|reload)
  sig HUP && echo reloaded OK && exit 0
  echo >&2 "Couldn't reload, starting '$CMD' instead"
  su - rails -c "$CMD"
  ;;
upgrade)
  if sig USR2 && sleep 20 && sig 0 && oldsig QUIT
  then
    n=$TIMEOUT
    while test -s $old_pid && test $n -ge 0
    do
      printf '.' && sleep 1 && n=$(( $n - 1 ))
    done
    echo

    if test $n -lt 0 && test -s $old_pid
    then
      echo >&2 "$old_pid still exists after $TIMEOUT seconds"
      exit 1
    fi
    exit 0
  fi
  echo >&2 "Couldn't upgrade, starting '$CMD' instead"
  su - rails -c "$CMD"
  ;;
kill_worker)
  workersig QUIT $2 && exit 0
  echo >&2 "Worker not running"
  ;;

reopen-logs)
  sig USR1
  ;;
*)
  echo >&2 "Usage: $0 <start|stop|restart|upgrade|force-stop|reopen-logs>"
  exit 1
  ;;
esac

unicorn.rb

APP_ROOT = File.expand_path(File.dirname(File.dirname(__FILE__)))

if ENV['MY_RUBY_HOME'] && ENV['MY_RUBY_HOME'].include?('rvm')
  begin
    rvm_path = File.dirname(File.dirname(ENV['MY_RUBY_HOME']))
    rvm_lib_path = File.join(rvm_path, 'lib')
    $LOAD_PATH.unshift rvm_lib_path
    require 'rvm'
    RVM.use_from_path! APP_ROOT
  rescue LoadError
    raise "RVM ruby lib is currently unavailable."
  end
end

ENV['BUNDLE_GEMFILE'] = File.expand_path('../Gemfile', File.dirname(__FILE__))
require 'bundler/setup'

worker_processes 1
working_directory APP_ROOT

preload_app true

timeout 30

listen APP_ROOT + "/tmp/sockets/unicorn.sock", :backlog => 64

pid APP_ROOT + "/tmp/pids/unicorn.pid"

stderr_path APP_ROOT + "/log/unicorn.stderr.log"
stdout_path APP_ROOT + "/log/unicorn.stdout.log"

before_fork do |server, worker|
  defined?(ActiveRecord::Base) && ActiveRecord::Base.connection.disconnect!

  old_pid = Rails.root + '/tmp/pids/unicorn.pid.oldbin'
  if File.exists?(old_pid) && server.pid != old_pid
    begin
      Process.kill("QUIT", File.read(old_pid).to_i)
    rescue Errno::ENOENT, Errno::ESRCH
      puts "Old master alerady dead"
    end
  end
end

after_fork do |server, worker|
  defined?(ActiveRecord::Base) && ActiveRecord::Base.establish_connection
  child_pid = server.config[:pid].sub('.pid', ".#{worker.nr}.pid")
  system("echo #{Process.pid} > #{child_pid}")
end

unicorn_worker_0.monitrc

check process unicorn_worker_0
with pidfile /path/to/your/app/shared/pids/unicorn.0.pid
start program = "/bin/cat /dev/null"
stop program = "/etc/init.d/unicorn kill_worker 0"
if mem is greater than 175.0 MB for 1 cycles then restart
if cpu is greater than 22% for 2 cycles then alert         
if cpu is greater than 25% for 1 cycles then restart

G'day George. I am just wondering in regards to your "upgrade" action, does it need to worry about killing off the PID from $old_pid, since the before_fork block inside the unicorn config is already handling that, in a much smarter manner (e.g. it checks to make sure pids aren't the same before killing).

In my experience, $old_pid is always identical to the pid already running, so having the upgrade action send a QUIT to $old_pid is less than ideal, and always results in failure. I simply have the upgrade action send a USR2 signal to the main PID and let the before_fork block you have written handle the rest.

@tigris I think you are right, although these script have been running in production with successful hot restarts. I'll need to run some tests locally. Thanks for the heads up!

Just a note, I've been running into something of a race condition on killing the old master with a similar setup.

Both the unicorn init script and the unicorn app script attempt to kill the old master. If the app kills it before the init script, then the init script upgrade will fail.

I figure it's better to just let the app kill off the old master since that guarantees that the new master is running the app.

The upgrade action can simply be:

upgrade)
    sig USR2 && exit 0
    $CMD
    ;;

This line has a typo (remove extra "m")
https://gist.github.com/shapeshed/1221753#file-unicorn-L33

Should look like:
case $action in
start)

Guys, there is little problem with using unicorn service under «rails» user. It requires root password because of su - rails -c "$CMD".

In my script I:

1. defined additional two vars

APP_USER="deployer"
CURRENT_USER=`whoami`

2. new function

exec_unicorn_cmd () {
  if [ "$CURRENT_USER" = $APP_USER ]
  then
    eval $CMD
  else
    su - $APP_USER -c "$CMD"
  fi
}

3. and using exec_unicorn_cmd instead su - rails -c "$CMD" (lines 35, 48, 68).

And it's works fine for all users. ;o)