Skip to content

Instantly share code, notes, and snippets.

@tomazzaman
Last active December 9, 2023 10:10
Show Gist options
  • Star 22 You must be signed in to star a gist
  • Fork 6 You must be signed in to fork a gist
  • Save tomazzaman/63265dfab3a9a61781993212fa1057cb to your computer and use it in GitHub Desktop.
Save tomazzaman/63265dfab3a9a61781993212fa1057cb to your computer and use it in GitHub Desktop.
Kill supervisor on Docker when any of the services fail

Killing supervisor if any of it's child processes fail

The trick is to only register the listener for events that indicate failure, namely

  • PROCESS_STATE_STOPPED
  • PROCESS_STATE_EXITED
  • PROCESS_STATE_FATAL

Once they do, we should send a SIGQUIT to Supervisor.

#!/bin/bash
printf "READY\n";
while read line; do
echo "Processing Event: $line" >&2;
kill -3 $(cat "/var/run/supervisord.pid")
done < /dev/stdin
[supervisord]
nodaemon=true
loglevel=debug
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid
childlogdir=/var/log/supervisor
[program:nginx]
command=nginx -g "daemon off;"
redirect_stderr=true
[program:php-fpm]
command=php-fpm
redirect_stderr=true
[eventlistener:processes]
command=stop-supervisor.sh
events=PROCESS_STATE_STOPPED, PROCESS_STATE_EXITED, PROCESS_STATE_FATAL
@sqqqrly
Copy link

sqqqrly commented Aug 21, 2019

You could also use supervisorctl shutdown. If you run shutdown from within the supervisorctl shell, you get prompted for confirmation. From a bash shell, there is no prompt and supervisord exits.

@albertoammar
Copy link

Here's a small improvement:

  • Use -SIGQUIT rather than hard-coding the signal name
  • Use $PPID to get the PID of supervisord (the parent of the event listener)
 while read line; do
   echo "Processing Event: $line" >&2;
-  kill -3 $(cat "/var/run/supervisord.pid")
+  kill -SIGQUIT $PPID
 done < /dev/stdin

Thanks, your save my soul

@clayrisser
Copy link

clayrisser commented Oct 17, 2020

For those who don't want a separate file 😄

[supervisord]
loglevel=warn
nodaemon=true

[program:hi]
command=bash -c "echo waiting 5 seconds . . . && sleep 5"
autorestart=false
numprocs=1
startsecs=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0

[eventlistener:processes]
command=bash -c "printf 'READY\n' && while read line; do kill -SIGQUIT $PPID; done < /dev/stdin"
events=PROCESS_STATE_STOPPED,PROCESS_STATE_EXITED,PROCESS_STATE_FATAL

@diemol
Copy link

diemol commented Feb 6, 2022

I've found the approach mentioned in this comment much more simple.
Supervisor/supervisor#712 (comment)

@MiguelSilvaViana
Copy link

I don't understand do I need to put this code inside the while loop?

echo "Processing Event: $line" >&2;
kill -3 $(cat "/var/run/supervisord.pid")

The browser needs nothing else and this will solve the memory error really, I believe it is necessary that the dead processes are killing the application when the browser consumes 1gb but just these two lines would already solve the problem?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment