Skip to content

Instantly share code, notes, and snippets.

@johnyvelho
Last active April 22, 2024 13:18
Show Gist options
  • Save johnyvelho/6a457d5e0500ca0c1faa2e6c5eab7ced to your computer and use it in GitHub Desktop.
Save johnyvelho/6a457d5e0500ca0c1faa2e6c5eab7ced to your computer and use it in GitHub Desktop.
Installing Supervisor on Elastic Beanstalk - 2021 - Linux AMI 2 - Laravel Worker Setup
  • Create the following folder structure in your root project directory: .ebextensions/supervisor

  • Place the supervisor.config under .ebextensions/

  • Place the setup.sh under .ebextensions/supervisor/

  • Run "chmod +x .ebextensions/supervisor/setup.sh"

  • Place the supervisord.conf under .ebextensions/supervisor/

  • Place the supervisor_laravel.conf under .ebextensions/supervisor/

Feel free to update the files according to your needs. You mostly will need to change the command inside the supervisor_laravel.conf file

If you want to check if the supervisor is running, connect to the instance and run "supervisorctl status". If you want stop/kill supervisor run "ps aux | grep "[/]usr/bin/supervisord", get the PID and then run "kill PID_NUMBER"

#!/bin/bash
# If you want to have more than one application, and in just one of them to run the supervisor, uncomment the lines below,
# and add the env variable IS_WORKER as true in the EBS application you want the supervisor
#if [ "${IS_WORKER}" != "true" ]; then
# echo "Not a worker. Set variable IS_WORKER=true to run supervisor on this instance"
# exit 0
#fi
echo "Supervisor - starting setup"
. /opt/elasticbeanstalk/deployment/env
if [ ! -f /usr/bin/supervisord ]; then
echo "installing supervisor"
easy_install supervisor
else
echo "supervisor already installed"
fi
if [ ! -d /etc/supervisor ]; then
mkdir /etc/supervisor
echo "create supervisor directory"
fi
if [ ! -d /etc/supervisor/conf.d ]; then
mkdir /etc/supervisor/conf.d
echo "create supervisor configs directory"
fi
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf
if ps aux | grep "[/]usr/bin/supervisord"; then
echo "supervisor is running"
else
echo "starting supervisor"
/usr/bin/supervisord
fi
/usr/bin/supervisorctl reread
/usr/bin/supervisorctl update
echo "Supervisor Running!"
container_commands:
01_install_supervisor:
command: ".ebextensions/supervisor/setup.sh"
[program:laravel-worker]
process_name=%(program_name)s_%(process_num)02d
command=/usr/bin/php /var/app/current/artisan queue:work sqs --sleep=3 --tries=3 --max-time=3600
autostart=true
autorestart=true
stopasgroup=true
killasgroup=true
;user=forge
numprocs=3
redirect_stderr=true
stderr_logfile=/var/log/supervisor_laravel.err.log
stdout_logfile=/var/log/supervisor_laravel.out.log
stopwaitsecs=3600
[unix_http_server]
file=/tmp/supervisor.sock ; (the path to the socket file)
[supervisord]
logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10 ; (num of main logfile rotation backups;default 10)
loglevel=info ; (log level;default info; others: debug,warn,trace)
pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false ; (start in foreground if true;default false)
minfds=1024 ; (min. avail startup file descriptors;default 1024)
minprocs=200 ; (min. avail process descriptors;default 200)
environment=SYMFONY_ENV=prod
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[include]
files = /etc/supervisor/conf.d/*.conf
[inet_http_server]
port = 9000
username = user
password = pw
@oulfr
Copy link

oulfr commented Jun 3, 2022

For me the script work fine except the .sh isnot runnable the solution with git i found it here:
https://stackoverflow.com/a/70428653/1010600:

According to the docs, Platform hooks need to be executable. Of note, this means they need to be executable according to git, because that's what Elastic Beanstalk uses to deploy.

You can check if they are executable via git ls-files -s .platform; you should see 100755 before any shell files in the output of this command. If you see 100644 before any of your shell files, run git add --chmod=+x -- .platform///*.sh to make them executable.

@alexander-schranz
Copy link

alexander-schranz commented Jun 29, 2022

Thank you for this gist. I created an example repository and updated it with the supervisor why its focus on symfony application it has the common things in it adjustments on vhosts, cronjobs, supervisor, migration tasks which can be used as example for every PHP application:

https://github.com/alexander-schranz/AWS-Elastic-Beanstalk-PHP-Symfony-Example

PS: I ahad to remove all . /opt/elasticbeanstalk/deployment/env else it did throw strange erros with my composer_options.

@racemshipper
Copy link

@johnyvelho Thanks mate for sharing!

@robbietorinit
Copy link

robbietorinit commented Aug 1, 2022

This is not working for me on elastic beanstalk for laravel - getting the below error

/usr/lib/python2.7/site-packages/supervisor-4.2.4-py2.7.egg/supervisor/options.py:475: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2022-08-01 11:54:23,110 P10068 [INFO] 'Supervisord is running as root and it is searching '
2022-08-01 11:54:23,110 P10068 [INFO] Error: No config file found at default paths (/usr/etc/supervisord.conf, /usr/supervisord.conf, supervisord.conf, etc/supervisord.conf, /etc/supervisord.conf, /etc/supervisor/supervisord.conf); use the -c option to specify a config file at a different path
2022-08-01 11:54:23,110 P10068 [INFO] For help, use /usr/bin/supervisord -h
2022-08-01 11:54:23,110 P10068 [INFO] error: <class 'socket.error'>, [Errno 111] Connection refused: file: /usr/lib64/python2.7/socket.py line: 575
2022-08-01 11:54:23,110 P10068 [INFO] error: <class 'socket.error'>, [Errno 111] Connection refused: file: /usr/lib64/python2.7/socket.py line: 575
2022-08-01 11:54:23,110 P10068 [INFO] Supervisor Running!

@johnyvelho please help?

@devrheber
Copy link

I'm getting the same error, could someone fix it?

This is not working for me on elastic beanstalk for laravel - getting the below error

/usr/lib/python2.7/site-packages/supervisor-4.2.4-py2.7.egg/supervisor/options.py:475: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security. 2022-08-01 11:54:23,110 P10068 [INFO] 'Supervisord is running as root and it is searching ' 2022-08-01 11:54:23,110 P10068 [INFO] Error: No config file found at default paths (/usr/etc/supervisord.conf, /usr/supervisord.conf, supervisord.conf, etc/supervisord.conf, /etc/supervisord.conf, /etc/supervisor/supervisord.conf); use the -c option to specify a config file at a different path 2022-08-01 11:54:23,110 P10068 [INFO] For help, use /usr/bin/supervisord -h 2022-08-01 11:54:23,110 P10068 [INFO] error: <class 'socket.error'>, [Errno 111] Connection refused: file: /usr/lib64/python2.7/socket.py line: 575 2022-08-01 11:54:23,110 P10068 [INFO] error: <class 'socket.error'>, [Errno 111] Connection refused: file: /usr/lib64/python2.7/socket.py line: 575 2022-08-01 11:54:23,110 P10068 [INFO] Supervisor Running!

@johnyvelho please help?

@adityar15
Copy link

It worked for me. Combining the solutions from @johnyvelho @omidnadeem @johnabelardom, it works fine. Thanks a lot :)

@collisiondetection
Copy link

It worked for me. Combining the solutions from @johnyvelho @omidnadeem @johnabelardom, it works fine. Thanks a lot :)

@adityar15 Can you please show us your complete working version?

@aliGIThaider
Copy link

starting supervisor
2023-01-23 09:32:11,528 P3636 [INFO] /usr/lib/python2.7/site-packages/supervisor-4.2.5-py2.7.egg/supervisor/options.py:475: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
2023-01-23 09:32:11,528 P3636 [INFO] 'Supervisord is running as root and it is searching '
2023-01-23 09:32:11,528 P3636 [INFO] Error: No config file found at default paths (/usr/etc/supervisord.conf, /usr/supervisord.conf, supervisord.conf, etc/supervisord.conf, /etc/supervisord.conf, /etc/supervisor/supervisord.conf); use the -c option to specify a config file at a different path
2023-01-23 09:32:11,528 P3636 [INFO] For help, use /usr/bin/supervisord -h

facing same issue anyone got it fixed?

@aliGIThaider
Copy link

It worked for me. Combining the solutions from @johnyvelho @omidnadeem @johnabelardom, it works fine. Thanks a lot :)

@adityar15 Can you please show us your complete working version?

can you share your working version of files in .ebextensions

@gonzalo-massa
Copy link

If someone is struggling with the error "/bin/sh: .ebextensions/supervisor/setup.sh: /bin/bash^M: bad interpreter: No such file or directory" is because you're uploading using eb deploy from windows and the ZIP file encodes newlines in windows format (CRLF), note the "^M" after "bash".

To fix that, I've added new commands to supervisor.config

container_commands:
    01_fix_script_permissions:
        command: "chmod +x .ebextensions/supervisor/setup.sh"
    02_fix_newlines_a:
        command: "sed -i 's/^M$//' .ebextensions/supervisor/setup.sh"
    02_fix_newlines_b:
        command: "sed -i $'s/\r$//' .ebextensions/supervisor/setup.sh"
    03_install_supervisor:
        command: ".ebextensions/supervisor/setup.sh"

The first command adds the execution bit to the file in case it doesn't have it. The second (both a and b) removes those windows newlines (sincerely I don't know wich one fixes it, but I've left both and it works).

@gonzalo-massa
Copy link

Question about this three lines in setup.sh (31,32,33):

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

Why does all three start with . /opt/elasticbeanstalk/deployment/env && ?

I suppose they try to load the environment variables, but for what purpose? Is it neccessary?
It's throwing me an error (bash: shell: command not found) and I've removed that part (now all three start at cat ...) and works fine.

@ashiquepclilac
Copy link

above all .ebextenstion files have succesfully deployed to beanstalk.But i no log files (supervisor_laravel.err.log, supervisor_laravel.out.log) have created .Is that permission problem ?? if so means how can i add permisssion commands ?

@johnabelardom
Copy link

johnabelardom commented Sep 10, 2023

@fabioselau077
Copy link

fabioselau077 commented Sep 15, 2023

@aliGIThaider @collisiondetection @adityar15 @ashiquepclilac I created a git repo for the sample bundle I used in my project: https://github.com/johnabelardom/laravel-awseb-bundle

I deployed successfully but the sqs queue does not execute. I changed the command to command=/usr/bin/php /var/app/current/artisan queue:work sqs but to no avail

@johnabelardom
Copy link

@fabioselau077 sorry I haven’t worked with sqs before. Can you maybe try removing the database keyword on the artisan queue:work?

and try adding in the ENV QUEUE_CONNECTION=sqs

@adityar15
Copy link

With a recent AWS EBS instance, easy_install may not work. In that case, this modified script might help.

New setup.sh

#!/bin/bash

# If you want to have more than one application, and in just one of them to run the supervisor, uncomment the lines below,
# and add the env variable IS_WORKER as true in the EBS application you want the supervisor

#if [ "${IS_WORKER}" != "true" ]; then
#    echo "Not a worker. Set variable IS_WORKER=true to run supervisor on this instance"
#    exit 0
#fi

echo "Supervisor - starting setup"
. /opt/elasticbeanstalk/deployment/env

if [ ! -f /usr/local/bin/supervisord ]; then
    echo "installing supervisor"
    python3 -m ensurepip
    pip3 install supervisor
else
    echo "supervisor already installed"
fi

if [ ! -d /etc/supervisor ]; then
    mkdir /etc/supervisor
    echo "create supervisor directory"
fi

if [ ! -d /etc/supervisor/conf.d ]; then
    mkdir /etc/supervisor/conf.d
    echo "create supervisor configs directory"
fi

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

if ps aux | grep "[/]usr/bin/supervisord"; then
    echo "supervisor is running"
else
    echo "starting supervisor"
    /usr/local/bin/supervisord
fi

/usr/local/bin/supervisorctl reread
/usr/local/bin/supervisorctl update

echo "Supervisor Running!"

@javoscript
Copy link

Is this working for you at the moment? I get the files and supervisord daemon to start initially, but it exists after a minute (the whole supervisord process)...

@geraldarcega
Copy link

does anyone experienced having different SQS URL being use when supervisor process the SQS message? I think that the environment variables does not take effect.

@collisiondetection
Copy link

@aliGIThaider @collisiondetection @adityar15 @ashiquepclilac I created a git repo for the sample bundle I used in my project: https://github.com/johnabelardom/laravel-awseb-bundle

I deployed successfully but the sqs queue does not execute. I changed the command to command=/usr/bin/php /var/app/current/artisan queue:work sqs but to no avail

We used :
command=/usr/bin/php /var/www/html/artisan queue:work --sleep=3 --tries=20 ;
`

@collisiondetection
Copy link

above all .ebextenstion files have succesfully deployed to beanstalk.But i no log files (supervisor_laravel.err.log, supervisor_laravel.out.log) have created .Is that permission problem ?? if so means how can i add permisssion commands ?

Direct those logs to /var/log/php-fpm/www-error.log they will get sent to cloudwatch, then you can filter them out.

@johnabelardom
Copy link

With a recent AWS EBS instance, easy_install may not work. In that case, this modified script might help.

New setup.sh

#!/bin/bash

# If you want to have more than one application, and in just one of them to run the supervisor, uncomment the lines below,
# and add the env variable IS_WORKER as true in the EBS application you want the supervisor

#if [ "${IS_WORKER}" != "true" ]; then
#    echo "Not a worker. Set variable IS_WORKER=true to run supervisor on this instance"
#    exit 0
#fi

echo "Supervisor - starting setup"
. /opt/elasticbeanstalk/deployment/env

if [ ! -f /usr/local/bin/supervisord ]; then
    echo "installing supervisor"
    python3 -m ensurepip
    pip3 install supervisor
else
    echo "supervisor already installed"
fi

if [ ! -d /etc/supervisor ]; then
    mkdir /etc/supervisor
    echo "create supervisor directory"
fi

if [ ! -d /etc/supervisor/conf.d ]; then
    mkdir /etc/supervisor/conf.d
    echo "create supervisor configs directory"
fi

. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisor/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisord.conf > /etc/supervisord.conf
. /opt/elasticbeanstalk/deployment/env && cat .ebextensions/supervisor/supervisor_laravel.conf > /etc/supervisor/conf.d/supervisor_laravel.conf

if ps aux | grep "[/]usr/bin/supervisord"; then
    echo "supervisor is running"
else
    echo "starting supervisor"
    /usr/local/bin/supervisord
fi

/usr/local/bin/supervisorctl reread
/usr/local/bin/supervisorctl update

echo "Supervisor Running!"

@adityar15 Super thank you! I needed this one since I upgraded my EBS platforms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment