How to handle exit code 137 on docker

handle_code_137_docker.md

Problem

When you use docker with "multiple commands", you will write scripts like this:

run.sh

#!/bin/sh -e

pip install --upgrade poetry
poetry config settings.virtualenvs.create false
poetry install
python ./manage.py runserver 0.0.0.0:8000

backend.dockerfile

from python:3.6
env PYTHONUNBUFFERED 1
run mkdir /code
workdir /code

docker-compose.yml

version: '3'

services:
  db:
    image: postgres:alpine
    environment:
      POSTGRES_PASSWORD: fingine
      POSTGRES_USER: fingine
      POSTGRES_DB: fingine
  backend:
    build:
      context: ./docker
      dockerfile: backend.dockerfile
    command: bash -c ./run.sh
    stop_signal: SIGINT
    env_file: ./docker/dev.env
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

Then, running the container with docker-compose up, and shutting them down with docker-compose stop, you'll get an error code 137 like this: fingine_backend_1 exited with code 137. Code 137 means your app catched SIGKILL that means Unexpectedly Closed. This is not good for your apps and this article describes how to fix it.

How to fix

The point is exec built-in utility. In general, the commands in the shell script you wrote are executed in child process of the script. i.e. shell spawns new processes to execute your commands. This behavior means your application would not be able to receive signals. To avoid this behavior, you will need to replace shell's process with the command without creating a new process. This is what exec does. Therefore, you need to change run.sh to this:

run.sh

#!/bin/sh -e

pip install --upgrade poetry
poetry config settings.virtualenvs.create false
poetry install
exec python ./manage.py runserver 0.0.0.0:8000  # <-- This. Prepend exec before run command of your backend server.

However, This is not enough. You will still get code 137. Why? docker only sends the signal to PID1. This behavior is to make management easy for supervisor to handle signals. So, to make run.sh run PID1, you will need to change command field on docker-compose.yml to this:

docker-compose.yml

version: '3'

services:
  db:
    image: postgres:alpine
    environment:
      POSTGRES_PASSWORD: fingine
      POSTGRES_USER: fingine
      POSTGRES_DB: fingine
  backend:
    build:
      context: ./docker
      dockerfile: backend.dockerfile
    command: ['./run.sh'] # <-- This. you should re-write this line to exec-style and without running sh -c
    stop_signal: SIGINT
    env_file: ./docker/dev.env
    volumes:
      - .:/code
    ports:
      - "8000:8000"
    depends_on:
      - db

So, running the composer, and shutting the containers down, you will get: fingine_backend_1 exited with code 0. Code 0 means exit successful. Fantastic!!

Conclusion

I appreciate Mr.Schlawack's article and I'd like to say thanks that the article solves the problem on my docker apps.

Reference

Hynek Schlawack, Why Your Dockerized Application Isn’t Receiving Signals, 19 June 2017