public
Last active

NGINX+SPDY with Unicorn. True Zero-Downtime unless migrations. Best practices.

  • Download Gist
0. nginx_setup.sh
Shell
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46
# Nginx+Unicorn best-practices congifuration guide. Heartbleed fixed.
# We use latest stable nginx with fresh **openssl**, **zlib** and **pcre** dependencies.
# Some extra handy modules to use: --with-http_stub_status_module --with-http_gzip_static_module
#
# Deployment structure
#
# SERVER:
# /etc/init.d/nginx (1. nginx)
# /home/app/public_html/app_production/current (Capistrano directory)
#
# APP:
# config/server/production/nginx.conf (2. nginx.conf)
# config/server/production/nginx_host.conf (3. nginx_host.conf)
# config/server/production/nginx_errors.conf (4. nginx_errors.conf)
# config/deploy.rb (5. deploy.rb)
# config/deploy/production.rb (6. production.rb)
# config/server/production/unicorn.rb (7. unicorn.rb)
# config/server/production/unicorn_init.sh (8. unicorn_init.sh)
 
 
 
 
 
 
cd /usr/src
wget http://nginx.org/download/nginx-1.5.13.tar.gz
tar xzvf ./nginx-1.5.13.tar.gz && rm -f ./nginx-1.5.13.tar.gz
 
wget ftp://ftp.csx.cam.ac.uk/pub/software/programming/pcre/pcre-8.32.tar.gz
tar xzvf pcre-8.32.tar.gz && rm -f ./pcre-8.32.tar.gz
 
wget http://www.openssl.org/source/openssl-1.0.1g.tar.gz
tar xzvf openssl-1.0.1g.tar.gz && rm -f openssl-1.0.1g.tar.gz
 
cd nginx-1.5.13
./configure --prefix=/opt/nginx --with-pcre=/usr/src/pcre-8.32 --with-openssl-opt=no-krb5 --with-openssl=/usr/src/openssl-1.0.1g --with-http_ssl_module --with-http_spdy_module --without-mail_pop3_module --without-mail_smtp_module --without-mail_imap_module --with-http_stub_status_module --with-http_gzip_static_module
 
make && make install
 
mkdir /tmp/client_body_temp
mkdir /opt/nginx/ssl_certs
 
echo "include /home/app/public_html/app_production/current/config/server/production/nginx.conf;" > /opt/nginx/conf/nginx.conf
 
vim /etc/init.d/nginx # see the config below
chmod +x /etc/init.d/nginx && update-rc.d -f nginx defaults
1. nginx
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65
#! /bin/sh
 
### BEGIN INIT INFO
# Provides: nginx
# Required-Start: $all
# Required-Stop: $all
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: starts the nginx web server
# Description: starts nginx using start-stop-daemon
### END INIT INFO
#
# /etc/init.d/nginx
 
 
PATH=/opt/nginx/sbin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/opt/nginx/sbin/nginx
NAME=nginx
DESC=nginx
 
test -x $DAEMON || exit 0
 
# Include nginx defaults if available
if [ -f /etc/default/nginx ] ; then
. /etc/default/nginx
fi
 
set -e
 
case "$1" in
start)
echo -n "Starting $DESC: "
start-stop-daemon --start --quiet --pidfile /opt/nginx/logs/$NAME.pid \
--exec $DAEMON -- $DAEMON_OPTS
echo "$NAME."
;;
stop)
echo -n "Stopping $DESC: "
start-stop-daemon --stop --quiet --pidfile /opt/nginx/logs/$NAME.pid \
--exec $DAEMON
echo "$NAME."
;;
restart|force-reload)
echo -n "Restarting $DESC: "
start-stop-daemon --stop --quiet --pidfile \
/opt/nginx/logs/$NAME.pid --exec $DAEMON
sleep 1
start-stop-daemon --start --quiet --pidfile \
/opt/nginx/logs/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS
echo "$NAME."
;;
reload)
echo -n "Reloading $DESC configuration: "
start-stop-daemon --stop --signal HUP --quiet --pidfile /opt/nginx/logs/$NAME.pid \
--exec $DAEMON
echo "$NAME."
;;
*)
N=/etc/init.d/$NAME
echo "Usage: $N {start|stop|restart|reload|force-reload}" >&2
exit 1
;;
esac
 
exit 0
2. nginx.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60
# Nginx main block configuration file.
# The most important directives here are ssl_protocols and ssl_ciphers
# Keep nginx configuration in repo, then just include it in /opt/nginx/conf/nginx.conf
# And DDoS prevent attack with directive limit_req_zone (limit 10 request/sec from 1 IP address)
# then it enables in the server block by "limit_req zone=one".
#
# config/server/production/nginx.conf
 
 
user app;
worker_processes 2;
worker_priority -5;
timer_resolution 100ms;
 
error_log logs/nginx.error.log;
 
events {
use epoll;
worker_connections 2048;
}
 
 
http {
client_max_body_size 25m;
client_body_buffer_size 128k;
client_body_temp_path /tmp/client_body_temp;
 
include mime.types;
default_type application/octet-stream;
server_tokens off;
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 70;
 
gzip on;
gzip_http_version 1.1;
gzip_disable "msie6";
gzip_vary on;
gzip_min_length 1100;
gzip_buffers 64 8k;
gzip_comp_level 3;
gzip_proxied any;
gzip_types text/plain text/css application/x-javascript text/xml application/xml;
 
ssl_certificate /opt/nginx/ssl_certs/server.crt;
ssl_certificate_key /opt/nginx/ssl_certs/server.key;
ssl_session_timeout 15m;
ssl_protocols SSLv3 TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_session_cache shared:SSL:10m;
ssl_stapling on;
 
add_header Strict-Transport-Security "max-age=16070400; includeSubdomains";
add_header X-Frame-Options DENY;
 
limit_req_zone $binary_remote_addr zone=one:10m rate=50r/s;
 
include /home/app/public_html/app_production/current/config/server/nginx_host.conf;
}
3. nginx_host.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57
# Nginx server block configuration with proxy_pass to Unicorn upstream
# We use full-SSL site with web-server redirection, no mess with Rails application redirection
#
# config/server/production/nginx_host.conf
 
 
upstream unicorn {
server unix:/tmp/unicorn.production.sock fail_timeout=0;
}
 
 
server {
listen 80;
server_name server.com;
rewrite ^(.*) https://$host$1 permanent;
 
location ~ \.(php|html)$ {
deny all;
}
 
access_log /dev/null;
error_log /dev/null;
}
 
 
server {
ssl on;
listen 443 spdy ssl;
server_name server.com;
root /home/app/public_html/app_production/current/public;
try_files $uri /system/maintenance.html @unicorn;
 
location @unicorn {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://unicorn;
limit_req zone=one burst=15;
 
access_log /dev/null;
error_log logs/unicorn.error.log;
}
 
location ~ ^/(assets|images|javascripts|stylesheets|swfs|system)/ {
gzip_static on;
expires max;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
}
 
include /home/app/public_html/app_production/current/config/server/production/nginx_errors.conf;
 
access_log /dev/null;
error_log /dev/null;
}
4. nginx_errors.conf
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34
# nginx configuration piece to handle errorrs
#
# config/server/production/nginx_errors.conf
 
error_page 500 502 504 /500.html;
error_page 503 @503;
 
location = /50x.html {
root html;
}
 
location = /404.html {
root html;
}
 
location @503 {
error_page 405 = /system/maintenance.html;
if (-f $document_root/system/maintenance.html) {
rewrite ^(.*)$ /system/maintenance.html break;
}
rewrite ^(.*)$ /503.html break;
}
 
if ($request_method !~ ^(GET|HEAD|PUT|POST|DELETE|OPTIONS)$ ){
return 405;
}
 
if (-f $document_root/system/maintenance.html) {
return 503;
}
 
location ~ \.(php|html)$ {
return 405;
}
5. deploy.rb
Ruby
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115
# Capistrano configuration. Now with TRUE zero-downtime unless DB migration.
#
# require 'new_relic/recipes' - Newrelic notification about deployment
# require 'capistrano/ext/multistage' - We use 2 deployment environment: staging and production.
# set :deploy_via, :remote_cache - fetch only latest changes during deployment
# set :normalize_asset_timestamps - no need to touch (date modification) every assets
# "deploy:web:disable" - traditional maintenance page (during DB migrations deployment)
# task :restart - Unicorn with preload_app should be reloaded by USR2+QUIT signals, not HUP
#
# http://unicorn.bogomips.org/SIGNALS.html
# "If “preload_app” is true, then application code changes will have no effect;
# USR2 + QUIT (see below) must be used to load newer code in this case"
#
# config/deploy.rb
 
 
require 'bundler/capistrano'
require 'capistrano/ext/multistage'
require 'new_relic/recipes'
 
set :stages, %w(staging production)
set :default_stage, "staging"
 
set :scm, :git
set :repository, "..."
set :deploy_via, :remote_cache
default_run_options[:pty] = true
 
set :application, "app"
set :use_sudo, false
set :user, "app"
set :normalize_asset_timestamps, false
 
 
before "deploy", "deploy:delayed_job:stop"
before "deploy:migrations", "deploy:delayed_job:stop"
 
after "deploy:update_code", "deploy:symlink_shared"
before "deploy:migrate", "deploy:web:disable", "deploy:db:backup"
 
after "deploy", "newrelic:notice_deployment", "deploy:cleanup", "deploy:delayed_job:restart"
after "deploy:migrations", "deploy:web:enable", "newrelic:notice_deployment", "deploy:cleanup", "deploy:delayed_job:restart"
 
 
namespace :deploy do
 
%w[start stop].each do |command|
desc "#{command} unicorn server"
task command, :roles => :app, :except => { :no_release => true } do
run "#{current_path}/config/server/#{rails_env}/unicorn_init.sh #{command}"
end
end
 
desc "restart unicorn server"
task :restart, :roles => :app, :except => { :no_release => true } do
run "#{current_path}/config/server/#{rails_env}/unicorn_init.sh upgrade"
end
 
 
desc "Link in the production database.yml and assets"
task :symlink_shared do
run "ln -nfs #{deploy_to}/shared/config/database.yml #{release_path}/config/database.yml"
end
 
namespace :delayed_job do
desc "Restart the delayed_job process"
task :restart, :roles => :app, :except => { :no_release => true } do
run "cd #{current_path}; RAILS_ENV=#{rails_env} bundle exec script/delayed_job restart" rescue nil
end
desc "Stop the delayed_job process"
task :stop, :roles => :app, :except => { :no_release => true } do
run "cd #{current_path}; RAILS_ENV=#{rails_env} bundle exec script/delayed_job stop" rescue nil
end
end
 
 
namespace :db do
desc "backup of database before migrations are invoked"
task :backup, :roles => :db, :only => { :primary => true } do
filename = "#{deploy_to}/shared/db_backup/#{stage}_db.#{Time.now.utc.strftime("%Y-%m-%d_%I:%M")}_before_deploy.gz"
text = capture "cat #{deploy_to}/current/config/database.yml"
yaml = YAML::load(text)["#{stage}"]
on_rollback { run "rm #{filename}" }
run "mysqldump --single-transaction --quick -u#{yaml['username']} -h#{yaml['host']} -p#{yaml['password']} #{yaml['database']} | gzip -c > #{filename}"
end
end
 
 
namespace :web do
desc "Maintenance start"
task :disable, :roles => :web do
on_rollback { run "rm #{shared_path}/system/maintenance.html" }
page = File.read("public/503.html")
put page, "#{shared_path}/system/maintenance.html", :mode => 0644
end
desc "Maintenance stop"
task :enable, :roles => :web do
run "rm #{shared_path}/system/maintenance.html"
end
end
 
end
 
 
namespace :log do
desc "A pinch of tail"
task :tailf, :roles => :app do
run "tail -n 10000 -f #{shared_path}/log/#{rails_env}.log" do |channel, stream, data|
puts "#{data}"
break if stream == :err
end
end
end
6. production.rb
Ruby
1 2 3 4 5 6 7 8 9
# capistrano production config
#
# config/deploy/production.rb
 
 
server "8.8.8.8", :app, :web, :db, :primary => true
set :branch, "production"
set :deploy_to, "/home/app/public_html/app_production"
set :rails_env, "production"
7. unicorn.rb
Ruby
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56
# Unicorn configuration file to be running by unicorn_init.sh with capistrano task
# read an example configuration before: http://unicorn.bogomips.org/examples/unicorn.conf.rb
#
# working_directory, pid, paths - internal Unicorn variables must to setup
# worker_process 4 - is good enough for serve small production application
# timeout 30 - time limit when unresponded workers to restart
# preload_app true - the most interesting option that confuse a lot of us,
# just setup is as true always, it means extra work on
# deployment scripts to make it correctly
# BUNDLE_GEMFILE - make Gemfile accessible with new master
# before_fork, after_fork - reconnect to all dependent services: DB, Redis, Sphinx etc.
# deal with old_pid only if CPU or RAM are limited enough
#
# config/server/production/unicorn.rb
 
 
app_path = "/home/app/public_html/app_production/current"
 
working_directory "#{app_path}"
pid "#{app_path}/tmp/pids/unicorn.pid"
stderr_path "#{app_path}/log/unicorn.log"
stdout_path "#{app_path}/log/unicorn.log"
 
listen "/tmp/unicorn.production.sock"
worker_processes 4
timeout 30
preload_app true
 
 
before_exec do |server|
ENV["BUNDLE_GEMFILE"] = "#{app_path}/Gemfile"
end
 
 
before_fork do |server, worker|
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
end
 
if defined?(Resque)
Resque.redis.quit
end
 
sleep 1
end
 
 
after_fork do |server, worker|
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
end
if defined?(Resque)
Resque.redis = 'localhost:6379'
end
end
8. unicorn_init.sh
Shell
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
# Unicorn handle shell script
#
# APP_ROOT, PID - are the same as you setup above
# CMD - use bundle binstubs (bundle install --binstubs) to
# forget about "bundle exec" stuff, run in demonize mode
# bin/unicorn is for Rack application (config.ru in root dir), but
# bin/unicorn_rails is to use with Rails 2.3
#
# To handle "app_preload true" configuration we should use USR2+QUIT signals, not HUP!
# So we rewrite capistrano deployment scripts to manage it.
#
# config/server/production/unicorn_init.sh
 
 
#!/bin/sh
set -e
# Example init script, this can be used with nginx, too,
# since nginx and unicorn accept the same signals
 
TIMEOUT=${TIMEOUT-60}
APP_ROOT=/home/app/public_html/app_production/current
PID=$APP_ROOT/tmp/pids/unicorn.pid
CMD="$APP_ROOT/bin/unicorn -D -c $APP_ROOT/config/server/unicorn.rb -E production"
action="$1"
set -u
 
old_pid="$PID.oldbin"
 
cd $APP_ROOT || exit 1
 
sig () {
test -s "$PID" && kill -$1 `cat $PID`
}
 
oldsig () {
test -s $old_pid && kill -$1 `cat $old_pid`
}
 
case $action in
start)
sig 0 && echo >&2 "Already running" && exit 0
$CMD
;;
stop)
sig QUIT && exit 0
echo >&2 "Not running"
;;
force-stop)
sig TERM && exit 0
echo >&2 "Not running"
;;
restart|reload)
sig HUP && echo reloaded OK && exit 0
echo >&2 "Couldn't reload, starting '$CMD' instead"
$CMD
;;
upgrade)
if sig USR2 && sleep 15 && sig 0 && oldsig QUIT
then
n=$TIMEOUT
while test -s $old_pid && test $n -ge 0
do
printf '.' && sleep 1 && n=$(( $n - 1 ))
done
echo
 
if test $n -lt 0 && test -s $old_pid
then
echo >&2 "$old_pid still exists after $TIMEOUT seconds"
exit 1
fi
exit 0
fi
echo >&2 "Couldn't upgrade, starting '$CMD' instead"
$CMD
;;
reopen-logs)
sig USR1
;;
*)
echo >&2 "Usage: $0 <start|stop|restart|upgrade|force-stop|reopen-logs>"
exit 1
;;
esac

отличный конфиг, букмаркаю, nginx + unicorn, шустрая тема

UPDATE Nginx caching open file descriptors (static assets only), their sizes and modification times with using of open_file_cache

UPDATE unicorn USR2 with followed by sleep for 15sec of 2. This time is enough to preload the application.

при выполнении upgrade поулчаю следующее:

Couldn't upgrade, starting '/home/vagrant/app/current/bin/unicorn_rails -D -c /home/vagrant/app/shared/config/unicorn.rb -E production' instead
master failed to start, check stderr log for details

В логах следующее:

I, [2012-09-12T15:31:02.127414 #1417]  INFO -- : executing ["/home/vagrant/app/current/bin/unicorn_rails", "-D", "-c", "/home/vagrant/app/shared/config/unicorn.rb", "-E", "production", {5=>#<Kgio::UNIXServer:/home/vagrant/app/shared/tmp/sockets/unicorn.sock>}] (in /home/vagrant/app/releases/1)
/usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- bundler/setup (LoadError)
from /usr/local/rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from /home/vagrant/app/current/bin/unicorn_rails:14:in `<main>'

В чем может быть проблема?

@alfuken gem install bundler и проверь, что свежий Gemfile.lock в репо

UPDATE: now with SPDY support

@mikhailov, any reason for running Unicorn processes as root? Thanks for sharing.

@xymbol master process runs as root, but workers as app

The init.d scripts start nginx and unicorn but cannot perform the other actions (e.g. stop). Not sure if this is Ubuntu specific or not, but start-stop-daemon will not create a pid file without the flag --make-pid - and even then will write the wrong pid due to the nginx/unicorn processes forking from the main process.

Здравствуйте. Сделал как написано в вашем конфиге, но у меня не стартуют рельсы.
В логах nginx все норм, а вот в логах unicorn.error.log данное сообщение

2013/03/26 15:49:41 [error] 25371#0: *8 limiting requests, excess: 1.000 by zone "one", client: 192.168.91.185, server: localhost, request: "GET / HTTP/1.1", host: "192.168.91.15"

А сама страница отдает 404 Ошибку

Very good. Please change the comment: (limit 10 request/sec from 1 IP address) to (limit 50...) about nginx limit_req_zone.

@k3NGuru по-моему, limit_req_zone был поломан и его починили только несколько месяцев назад, пробуйте стартануть nginx без него

@mikhailov Все равно не заводится система. Уже сделал как написано тут https://coderwall.com/p/8igwqa настроил как написано в данном конфиге, Вот моя папка https://github.com/k3NGuru/mlvz

2013/04/05 16:02:07 [error] 19467#0: *2 limiting requests, excess: 1.000 by zone "one", client: 192.168.91.163, server: 192.168.91.15, request: "GET / HTTP/1.1^@^Dho^@^@", host: "192.168.91.15"

192.168.91.15 - Сервер
192.168.91.163 - Клиент

SSL настроил как написано тут http://wiki.nginx.org/HttpSslModule

@k3NGuru все дело в специфичной работе встроенного модуля limit_req, добавь burst=5 для решения проблемы:

limit_req         zone=one burst=5;

Это пропустит 6 запросов в секунду, надо больше с одной айпи - увеличивай burst, либо распараллель нагрузку между несколькими Nginx серверами. Почему так, а не иначе, попытались объяснить здесь: http://www.lexa.ru/nginx-ru/msg38569.html Сегодня задан вопрос в nginx-рассылку.

Thank you for sharing this. :-)

Amazing. Thank you very much man =)

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.