Node on Production

node_production.md

Running a node application on a local machine is easy - you just execute node app.js or, if you want to ease the development process, you use nodemon or forever. Things get a little bit more complicated when you decide to go live with the app and want it stable on the production server. This post describes how to get your application running as a service on CentOS Linux using init scripts and monit for managing and monitoring it.

Register your app as service - init.d

Let's say you have a nodejs application located on your server in /var/www/myapp and the main script is index.js.

First thing to do is to add your app to init scripts. To do so, create a new file in /etc/init.d and name it after your application.

/etc/init.d/myapp:

#!/bin/bash

### BEGIN INIT INFO
# Provides:          myapp
# Required-Start:    $syslog $remote_fs
# Required-Stop:     $syslog $remote_fs
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Starting and stopping myapp
# Description:       What does the app do
### END INIT INFO

RETVAL=0
NAME="myapp"
DIR="/var/www/myapp"
INIT="index.js"
LOG_FILE="$DIR/$NAME.log"
ENV_FILE="$DIR/env"

NODE=/usr/bin/node
PID_FILE="/var/run/$NAME.pid"

test -x $NODE || exit 0

function start_prod {
  start "production"
}

function start_dev {
  start "development"
}

function start {
  NODE_ENV=$1 nohup "$NODE" "$DIR/$INIT" 1>>"$LOG_FILE" 2>&1 &
  echo $1 > "$ENV_FILE"
  echo $! > "$PID_FILE"
  RETVAL=$?
}

function stop_app {
  kill `cat $PID_FILE`
  RETVAL=1
  RETVAL=$?
}

function restart_app {
  APP_ENV=`cat $ENV_FILE`
  stop_app
  start $APP_ENV
}

case $1 in
   start)
      start_prod ;;
    stop)
      stop_app ;;
    start_dev)
      start_dev ;;
    restart)
      restart_app ;;
    *)
      echo "usage: $NAME {start|stop|restart|start_dev}" ;;
esac
exit 0

To be able to run the script change the permissions: chmod +x myapp.

Now try the script by service myapp start.

Check if it's running: ps aux | grep node.

Run service myapp stop to stop the application.

If you wanted to start the app at system boot you could run chkconfig --add myapp, but don't. We'll leave this task to be handled by monit, and adding our app to chkconfig list could lead to some unwanted effects:

If monit is used to monitor services that are also started at boot time (e.g. services started via SYSV init rc scripts or via inittab) then in some situations a special race condition can occur. That is; if a service is slow to start, monit can assume that the service is not running and possibly try to start it and raise an alert, while, in fact the service is already about to start or already in its startup sequence.

So make sure your app is not listed chkconfig --list, and if it is, run chkconfig myapp --del.

To check all the available parameter type service myapp.

The start command runs the application in production mode (NODE_ENV=production), while start_dev sets the mode to development. The restart command reboots the process in the environment it has been started with.

Introducing Monit

From the documentation:

monit is a utility for managing and monitoring processes, programs, files, directories and filesystems on a Unix system. [...] Monit is primarily designed to run as a daemon process.

The primary task of Monit is to start processes and keep them running. By default Monit will check the proceess (your app) every 60 seconds, and if it's not runnint it'll try to wake it up.

Further on:

Monit can start a process if it does not run, restart a process if it does not respond and stop a process if it uses too much resources. [...] Monit can also monitor process characteristics, such as how much memory or cpu cycles a process is using.

Monit can be used from CLI as well as via http webpage. To enable the latter, open the configuration file /etc/monit.conf and uncomment the following lines:

 set httpd port 2812 and
     use address localhost
     # allow localhost
     allow admin:monit
     allow @monit
     allow @users readonly

You can change your credentials here and allow only specific IP addresses for access. Also if Monit is installed on a remote server you will have to replace localhost with your machine's IP address.

Start Monit deamon: service monit start and make it bootable on system start: chkconfig monit on

go to your domain:2812 (or add to nginx)

Add process to Monit: make new file /etc/monit.d/myapp.conf:

#Monitor myapp#
check process clippings with pidfile /var/run/myapp.pid
start program = "/etc/init.d/myapp start"
stop program  = "/etc/init.d/myapp stop"
if 5 restarts within 5 cycles then unmonitor

The first line check if the process is running under the pid in the pid file. The last line tells the Monit to stop monitoring after 5 unsuccesfull restarts.

For more information about Monit options and syntax consult the documentation.

After changing the service's settings you have to refresh Monit by calling monit reload myapp or monit restart myapp to restart it.

Now check (via http or terminal) monit status. You should see that our myapp process is monitored but not started 'Does not exist'.

To kick the new service off run monit start myapp.

Now, if you kill the process or call service myapp stop Monit will wake it up on the next cycle. To stop the service use monit stop myapp.

Also, if your server reboots the application should be automatically brought into life, although this can take a few minutes.