Skip to content

Instantly share code, notes, and snippets.

@StefanieD
Created July 3, 2015 15:17
Show Gist options
  • Star 30 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save StefanieD/5407df6b086b38c4f887 to your computer and use it in GitHub Desktop.
Save StefanieD/5407df6b086b38c4f887 to your computer and use it in GitHub Desktop.
Install and use Supervisord with AWS Elastic Beanstalk, Symfony 2 and RabbitMq Bundle
  • Place the supervisord.conf under .ebextensions/supervisor/

  • Place the supervisor.config under .ebextensions/

  • Place the data_import_consumer.conf under .ebextensions/supervisor/

[program:data_import_consumer]
process_name= %(process_num)s
command= /usr/bin/php /var/app/current/api/app/console rabbitmq:multiple-consumer -m 30 import_data
autostart=true
autorestart=unexpected
stderr_logfile=/var/log/app/rabbitmq/data_import_consumer.err.log
stdout_logfile=/var/log/app/rabbitmq/data_import_consumer.out.log
numprocs=3
exitcodes=1
startsecs=0
commands:
11install_supervisor:
test : '[ ! /etc/supervisor ] && echo "supervisor not installed"'
command: "easy_install supervisor"
container_commands:
14insert_supervisor_main_config:
command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf"
leader_only: true
15insert_supervisor_main_config_to_etc:
command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf"
leader_only: true
16insert_data_import_supervisor_config:
command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/data_import_consumer.conf > /etc/supervisor/conf.d/data_import_consumer.conf"
leader_only: true
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/490-supervisor_install.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
if [ ! -f /usr/local/bin/supervisord ]; then
echo "install supervisor and create directories"
easy_install supervisor
else
echo "supervisor already installed"
fi
if [ ! -d /etc/supervisor ]; then
mkdir /etc/supervisor
echo "create supervisor directory"
fi
if [ ! -d /etc/supervisor/conf.d ]; then
mkdir /etc/supervisor/conf.d
echo "create supervisor configs directory"
fi
"/opt/elasticbeanstalk/hooks/appdeploy/post/500-supervisor_update.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
. /opt/elasticbeanstalk/support/envvars
if ps aux | grep -q "[/]usr/local/bin/supervisord"; then
echo "supervisor is running"
else
echo "start supervisor"
/usr/bin/python /usr/local/bin/supervisord --pidfile /var/run/supervisord.pid
fi
supervisorctl reread
supervisorctl update
cd /var/app/current/api
app/console rabbitmq:setup-fabric --env=$SYMFONY_ENV
echo "$(date +'%Y%m%d %T') Check for supervisor update" >> /var/log/directory-hooks-executor.log
[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
environment=SYMFONY_ENV=prod
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[include]
files = /etc/supervisor/conf.d/*.conf
[inet_http_server]
port = 9000
username = user
password = pw
@buluxan
Copy link

buluxan commented Jun 10, 2016

hi thx for sharing , will try it on my elasticbeanstalk

thx u

@mellis3
Copy link

mellis3 commented Oct 5, 2017

When i try to use this in elastic beanstalk i get this error:

The configuration file __MACOSX/.ebextensions/._supervisor.config in application version XXXXX contains invalid YAML or JSON. YAML exception: Invalid Yaml: unacceptable character '�' (0x0) special characters are not allowed in "", position 0, JSON exception: Invalid JSON: Unexpected character (�) at position 0.. Update the configuration file.

I copied the code from the above code in the RAW format, so shouldn't have had any special characters in it.

Any thoughts?

Thanks!

@StefanieD
Copy link
Author

@mellis3 Sorry for the late answer but unfortunately Github doesn't send notifications for comments here.
Do you use any IDE, maybe the spaces were converted to tabs or something else. The script was working under ubuntu but it's also some time ago.
The only thing I can suggest is to debug it step by step. Comment out some parts and you'll find the wrong character.

@tvillafane
Copy link

With the new supervisor (version 3.0+ I think), the folder structures change. There is no longer a supervisor folder in /etc, so I had to change the following commands as such:
04_insert_supervisor_main_config: command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf" leader_only: true 05_insert_supervisor_main_config_to_etc: command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf" leader_only: true 06_insert_push_listener_supervisor_config: command: ". /opt/elasticbeanstalk/support/envvars && cat .ebextensions/supervisor/push_listener.conf > /etc/supervisor.conf" leader_only: true

Otherwise, everything else worked flawlessly. Thank you for this!

@jilster
Copy link

jilster commented Jul 27, 2018

@mellis3 You probably figured it out already by now, but maybe this will help someone else. This error is caused because of __MACOSX folders that are created automatically by MacOSX and shouldn't be in the zip you upload to the Elastic Beanstalk. To fix this, try:

zip -d Archive.zip "__MACOSX*"

(where Archive.zip is points to your zip)

@csimpi
Copy link

csimpi commented Jan 24, 2019

Thank you, this was very useful!

@futoase
Copy link

futoase commented May 15, 2019

Thank you this gist post!.

@kprosper
Copy link

please, how can i stop supervisor on deploy wity symfony ?

@StefanieD
Copy link
Author

@kprosper Maybe you could try to create a pre-deploy file i.e. /opt/elasticbeanstalk/hooks/appdeploy/pre/100-stop-supervisor.sh which stops supervisor.
Like I did it in supervisor.config for updating supervisor.

@danishmirza
Copy link

danishmirza commented Sep 4, 2020

I am getting this error

/bin/sh: /opt/elasticbeanstalk/support/envvars: No such file or directory

in amazon linux 2.

Any Replace of the line?

@StefanieD
Copy link
Author

@danishmirza

Hi danishmirza,
unfortunately this gist is very old. I think in the meantime AWS changed the structure in their Elastic Beanstalk instances. Do you have a chance to find out the directory for the environment variables on AWS instance?
Then you need to replace /opt/elasticbeanstalk/support/envvars with the current path.

Or you need to create the envvars file first.
See https://aws.amazon.com/premiumsupport/knowledge-center/view-environment-properties-elastic-beanstalk/

@StefanieD
Copy link
Author

@danishmirza
I just found an article in stackoverflow for Amazon Linux 2 instances. It's a little bit different from Amazon 1 instances.
You need to do something like this:

export $(cat /opt/elasticbeanstalk/deployment/env | xargs)

See https://stackoverflow.com/questions/63280543/aws-eb-deploy-doesnt-load-environment-variables

@kprosper
Copy link

Good morning everybody, thank you for your good work. please I want to ask how can i Restart supervisor main process which has started by /usr/bin/python /usr/local/bin/supervisord --pidfile /var/run/supervisord.pid. thank you for your answer.

@StefanieD
Copy link
Author

Hi @kprosper, you can create a new pre- or post-deploy script (see supervisor.config for examples)
In this script you can try to run service supervisor restart or
supervisorctl restart <name>

See here: https://www.onurguzel.com/supervisord-restarting-and-reloading/

@johnyvelho
Copy link

For those struggling to make it running in 2021: https://gist.github.com/johnyvelho/6a457d5e0500ca0c1faa2e6c5eab7ced

@StefanieD
Copy link
Author

@johnyvelho Cool, thanx for the update and your work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment