Install and use Supervisord with AWS Elastic Beanstalk, Symfony 2 and RabbitMq Bundle

Readme.md

• Place the supervisord.conf under .ebextensions/supervisor/

• Place the supervisor.config under .ebextensions/

• Place the data_import_consumer.conf under .ebextensions/supervisor/

data_import_consumer.conf

[program:data_import_consumer]
process_name= %(process_num)s
command= /usr/bin/php /var/app/current/api/app/console rabbitmq:multiple-consumer -m 30 import_data
autostart=true
autorestart=unexpected
stderr_logfile=/var/log/app/rabbitmq/data_import_consumer.err.log
stdout_logfile=/var/log/app/rabbitmq/data_import_consumer.out.log
numprocs=3
exitcodes=1
startsecs=0

supervisor.config

commands:
  11install_supervisor:
    test : '[ ! /etc/supervisor ] && echo "supervisor not installed"'
    command: "easy_install supervisor"


container_commands:
    14insert_supervisor_main_config:
        command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf"
        leader_only: true
    15insert_supervisor_main_config_to_etc:
        command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf"
        leader_only: true
    16insert_data_import_supervisor_config:
        command: ". /opt/elasticbeanstalk/support/envvars && cat  .ebextensions/supervisor/data_import_consumer.conf > /etc/supervisor/conf.d/data_import_consumer.conf"
        leader_only: true

files:
    "/opt/elasticbeanstalk/hooks/appdeploy/post/490-supervisor_install.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/usr/bin/env bash
            . /opt/elasticbeanstalk/support/envvars

            if [ ! -f /usr/local/bin/supervisord ]; then
                echo "install supervisor and create directories"
                easy_install supervisor
            else
                echo "supervisor already installed"
            fi

            if [ ! -d /etc/supervisor ]; then
                mkdir /etc/supervisor
                echo "create supervisor directory"
            fi

            if [ ! -d /etc/supervisor/conf.d ]; then
                mkdir /etc/supervisor/conf.d
                echo "create supervisor configs directory"
            fi
    "/opt/elasticbeanstalk/hooks/appdeploy/post/500-supervisor_update.sh":
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/usr/bin/env bash
            . /opt/elasticbeanstalk/support/envvars

            if ps aux | grep -q "[/]usr/local/bin/supervisord"; then
            echo "supervisor is running"
            else
            echo "start supervisor"
            /usr/bin/python /usr/local/bin/supervisord --pidfile /var/run/supervisord.pid
            fi

            supervisorctl reread
            supervisorctl update

            cd /var/app/current/api
            app/console rabbitmq:setup-fabric --env=$SYMFONY_ENV

            echo "$(date +'%Y%m%d %T') Check for supervisor update" >> /var/log/directory-hooks-executor.log

supervisord.conf

[unix_http_server]
file=/tmp/supervisor.sock   ; (the path to the socket file)

[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10           ; (num of main logfile rotation backups;default 10)
loglevel=info                ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false               ; (start in foreground if true;default false)
minfds=1024                  ; (min. avail startup file descriptors;default 1024)
minprocs=200                 ; (min. avail process descriptors;default 200)
environment=SYMFONY_ENV=prod

[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

[include]
files = /etc/supervisor/conf.d/*.conf

[inet_http_server]
port = 9000
username = user
password = pw

@mellis3 You probably figured it out already by now, but maybe this will help someone else. This error is caused because of __MACOSX folders that are created automatically by MacOSX and shouldn't be in the zip you upload to the Elastic Beanstalk. To fix this, try:

zip -d Archive.zip "__MACOSX*"

(where Archive.zip is points to your zip)

Thank you, this was very useful!

Thank you this gist post!.

please, how can i stop supervisor on deploy wity symfony ?

@kprosper Maybe you could try to create a pre-deploy file i.e. /opt/elasticbeanstalk/hooks/appdeploy/pre/100-stop-supervisor.sh which stops supervisor.
Like I did it in supervisor.config for updating supervisor.

I am getting this error

/bin/sh: /opt/elasticbeanstalk/support/envvars: No such file or directory

in amazon linux 2.

Any Replace of the line?

@danishmirza

Hi danishmirza,
unfortunately this gist is very old. I think in the meantime AWS changed the structure in their Elastic Beanstalk instances. Do you have a chance to find out the directory for the environment variables on AWS instance?
Then you need to replace /opt/elasticbeanstalk/support/envvars with the current path.

Or you need to create the envvars file first.
See https://aws.amazon.com/premiumsupport/knowledge-center/view-environment-properties-elastic-beanstalk/

@danishmirza
I just found an article in stackoverflow for Amazon Linux 2 instances. It's a little bit different from Amazon 1 instances.
You need to do something like this:

export $(cat /opt/elasticbeanstalk/deployment/env | xargs)

See https://stackoverflow.com/questions/63280543/aws-eb-deploy-doesnt-load-environment-variables

Good morning everybody, thank you for your good work. please I want to ask how can i Restart supervisor main process which has started by /usr/bin/python /usr/local/bin/supervisord --pidfile /var/run/supervisord.pid. thank you for your answer.

Hi @kprosper, you can create a new pre- or post-deploy script (see supervisor.config for examples)
In this script you can try to run service supervisor restart or
supervisorctl restart <name>

See here: https://www.onurguzel.com/supervisord-restarting-and-reloading/

For those struggling to make it running in 2021: https://gist.github.com/johnyvelho/6a457d5e0500ca0c1faa2e6c5eab7ced

@johnyvelho Cool, thanx for the update and your work.