Skip to content

Instantly share code, notes, and snippets.

@kbrwn

kbrwn/docker-failing-on-journald-exit

Last active October 5, 2015 23:29
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save kbrwn/2b9849e0e5a9823d6618 to your computer and use it in GitHub Desktop.
Save kbrwn/2b9849e0e5a9823d6618 to your computer and use it in GitHub Desktop.
Download ZIP
docker-failing-on-journald-exit
Raw
docker-failing-on-journald-exit
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0befefa8e7c debian "/bin/sh -c 'while t 6 seconds ago Up 5 seconds naughty_leakey
4252177ca338 fedora "/bin/sh -c 'while t 19 seconds ago Up 19 seconds naughty_kirch
fea64c8e213f ubuntu:14.04 "/bin/sh -c 'while t 3 minutes ago Up 3 minutes compassionate_perlman
kyle@journald-tester ~ $ sudo systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib64/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: active (running) since Mon 2015-10-05 20:50:06 UTC; 7min ago
Docs: http://docs.docker.com
Main PID: 3121 (docker)
Memory: 13.1M
CPU: 663ms
CGroup: /system.slice/docker.service
└─3121 docker --daemon --host=fd://
$ sudo systemctl status systemd-journald.service
● systemd-journald.service - Journal Service
Loaded: loaded (/usr/lib64/systemd/system/systemd-journald.service; static; vendor preset: disabled)
Active: active (running) since Mon 2015-10-05 20:50:03 UTC; 7min ago
Docs: man:systemd-journald.service(8)
man:journald.conf(5)
Main PID: 3101 (systemd-journal)
Status: "Processing requests..."
Memory: 956.0K
CPU: 242ms
CGroup: /system.slice/systemd-journald.service
└─3101 /usr/lib/systemd/systemd-journald
kyle@journald-tester ~/testbigdockerlogs $ date
Mon Oct 5 20:59:12 UTC 201
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0befefa8e7c debian "/bin/sh -c 'while t 6 seconds ago Up 5 seconds naughty_leakey
4252177ca338 fedora "/bin/sh -c 'while t 19 seconds ago Up 19 seconds naughty_kirch
fea64c8e213f ubuntu:14.04 "/bin/sh -c 'while t 3 minutes ago Up 3 minutes compassionate_perlman
kyle@journald-tester ~/testbigdockerlogs $ date
Mon Oct 5 20:59:12 UTC 2015
kyle@journald-tester ~/testbigdockerlogs $ sudo systemctl kill systemd-journald.service
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0befefa8e7c debian "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_leakey
4252177ca338 fedora "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_kirch
fea64c8e213f ubuntu:14.04 "/bin/sh -c 'while t 8 minutes ago Up 8 minutes compassionate_perlman
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0befefa8e7c debian "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_leakey
4252177ca338 fedora "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_kirch
fea64c8e213f ubuntu:14.04 "/bin/sh -c 'while t 8 minutes ago Up 8 minutes compassionate_perlman
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0befefa8e7c debian "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_leakey
4252177ca338 fedora "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_kirch
fea64c8e213f ubuntu:14.04 "/bin/sh -c 'while t 8 minutes ago Up 8 minutes compassionate_perlman
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
c0befefa8e7c debian "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_leakey
4252177ca338 fedora "/bin/sh -c 'while t 5 minutes ago Up 5 minutes naughty_kirch
fea64c8e213f ubuntu:14.04 "/bin/sh -c 'while t 8 minutes ago Up 8 minutes compassionate_perlman
kyle@journald-tester ~/testbigdockerlogs $ docker ps
Get http:///var/run/docker.sock/v1.19/containers/json: EOF. Are you trying to connect to a TLS-enabled daemon without TLS?
kyle@journald-tester ~/testbigdockerlogs $ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
kyle@journald-tester ~ $ sudo journalctl -f -u docker.service
-- Logs begin at Fri 2015-10-02 23:36:19 UTC. --
Oct 05 20:54:10 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:10.207933871Z" level=info msg="POST /v1.19/containers/7102082b2202/stop?t=10"
Oct 05 20:54:20 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:20.208928966Z" level=info msg="Container 7102082b22029f3bb2625fb6685af41a7601ab0c49da19ff3d26424e645506bd failed to exit within 10 seconds of SIGTERM - using the force"
Oct 05 20:54:20 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:20.226573069Z" level=info msg="POST /v1.19/containers/7102082b22029f3bb2625fb6685af41a7601ab0c49da19ff3d26424e645506bd/wait"
Oct 05 20:54:29 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:29.048681105Z" level=info msg="GET /v1.19/containers/json"
Oct 05 20:54:42 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:42.174989401Z" level=info msg="POST /v1.19/containers/create"
Oct 05 20:54:42 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:42.274942563Z" level=info msg="POST /v1.19/containers/4252177ca3382730fcd8addf7384bf366a126beffbe89ffe4900d452f343a7fd/start"
Oct 05 20:54:45 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:45.095879253Z" level=info msg="GET /v1.19/containers/json"
Oct 05 20:54:55 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:55.581613381Z" level=info msg="POST /v1.19/containers/create"
Oct 05 20:54:55 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:54:55.674645376Z" level=info msg="POST /v1.19/containers/c0befefa8e7cabbd5f9ee64e88cfbdb0c613e025a7f5e6b96d234f3585425942/start"
Oct 05 20:55:01 journald-tester.c.coreos-support.internal dockerd[3121]: time="2015-10-05T20:55:01.922037621Z" level=info msg="GET /v1.19/containers/json"
Oct 05 21:00:36 journald-tester.c.coreos-support.internal systemd[1]: Started Docker Application Container Engine.
Oct 05 21:00:36 journald-tester.c.coreos-support.internal systemd[1]: Starting Docker Application Container Engine...
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: time="2015-10-05T21:00:36.924088128Z" level=info msg="Loading containers: start."
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: ..............................................
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: time="2015-10-05T21:00:36.957862646Z" level=info msg="Loading containers: done."
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: time="2015-10-05T21:00:36.958137801Z" level=info msg="Daemon has completed initialization"
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: time="2015-10-05T21:00:36.958409752Z" level=info msg="Docker daemon" commit=df2f73d-dirty execdriver=native-0.2 graphdriver=overlay version=1.7.1
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: time="2015-10-05T21:00:36.976291985Z" level=info msg="Listening for HTTP on fd ()"
Oct 05 21:00:36 journald-tester.c.coreos-support.internal dockerd[5852]: time="2015-10-05T21:00:36.976740345Z" level=info msg="GET /v1.19/containers/json"
$ date
Mon Oct 5 21:02:35 UTC 2015
kyle@journald-tester ~ $ sudo journalctl -u systemd-journald
...
Oct 05 20:50:03 journald-tester.c.coreos-support.internal systemd-journal[3101]: Journal started
Oct 05 21:00:14 journald-tester.c.coreos-support.internal systemd-journal[3101]: Journal stopped
Oct 05 21:00:14 journald-tester.c.coreos-support.internal systemd-journal[5628]: Permanent journal is using 184.0M (max allowed 656.4M, trying to leave 984.7M free of 3.6G a
Oct 05 21:00:14 journald-tester.c.coreos-support.internal systemd-journal[5628]: Journal started
$ sudo strace -p 836 -s 80 -o dockerstracealpha.txt
Process 836 attached
kyle@alphajournaltest ~ $ cat dockerstracealpha.txt
read(46, <unfinished ...>
+++ killed by SIGPIPE +++
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment