logs-docker-oom-invalid-amount-of-swarm-replicas.md

logs-docker-oom-invalid-amount-of-swarm-replicas.md

Jan 06 14:04:15 master dockerd[2325]: time="2017-01-06T14:04:13.208839706Z" level=info msg="libcontainerd: new containerd process, pid: 4140"
Jan 06 14:04:17 master dockerd[2325]: time="2017-01-06T14:04:13.273558318Z" level=error msg="libcontainerd: failed to receive event from containerd: rpc error: code = 13 desc = transport is closing"
Jan 06 14:04:24 master dockerd[2325]: time="2017-01-06T14:04:22.526231387Z" level=info msg="libcontainerd: new containerd process, pid: 4141"
Jan 06 14:04:34 master kernel: gmain invoked oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0
.. snip ..
Jan 06 14:04:34 master kernel: Out of memory: Kill process 4132 (php) score 935 or sacrifice child
Jan 06 14:04:34 master kernel: Killed process 4132 (php) total-vm:3392804kB, anon-rss:2912200kB, file-rss:0kB
Jan 06 14:04:34 master dockerd[2325]: time="2017-01-06T14:04:34.595231149Z" level=info msg="libcontainerd: new containerd process, pid: 4145"
Jan 06 14:04:34 master dockerd[2325]: time="2017-01-06T14:04:34.633952118Z" level=warning msg="failed to retrieve containerd version: rpc error: code = 14 desc = grpc: the connection is unavailable"
Jan 06 14:04:34 master dockerd[2325]: time="2017-01-06T14:04:34.827790956Z" level=warning msg="failed to retrieve containerd version: rpc error: code = 14 desc = grpc: the connection is unavailable"
Jan 06 14:04:34 master dockerd[2325]: time="2017-01-06T14:04:34.853056460Z" level=error msg="agent: session failed" error="rpc error: code = 4 desc = context deadline exceeded" module="node/agent"
Jan 06 14:04:34 master dockerd[2325]: time="2017-01-06T14:04:34.862858532Z" level=error msg="containerd: deleting container" error="exit status 1: \"container 78b3797327b06614e4660ecba12ecf529de1e9c3d20ab644b5df1b738c855f22 does not exist\\none or more of the container deletions failed\\n\""
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2442]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.223576204Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/4d1b5a107a99003f35f4e1a28e0c325c6226e71dde5e202d8a2ba1fabd572bd3-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2703]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.252693792Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/96646e55706458468e98038a20bdb79361b4f577e960f9da520c5249a4552c2e-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.278525301Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/4d1b5a107a99003f35f4e1a28e0c325c6226e71dde5e202d8a2ba1fabd572bd3: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2442]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.284685799Z" level=info msg="libcontainerd: new containerd process, pid: 4187"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2703]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.298849226Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/67eb5f4dd293db24e093e2cbc3f466714eb560d2b09d277a11898dd5a234893b-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2442]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.321111899Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/96646e55706458468e98038a20bdb79361b4f577e960f9da520c5249a4552c2e: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2442]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.340697607Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/67eb5f4dd293db24e093e2cbc3f466714eb560d2b09d277a11898dd5a234893b: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[2890]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.358209655Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/03d3cb812c38fb1de38fc6ad1a5a4e890328380dcd8d667b48bb790e0eb34e9f-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master dockerd[2325]: time="2017-01-06T14:04:35.378726398Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/03d3cb812c38fb1de38fc6ad1a5a4e890328380dcd8d667b48bb790e0eb34e9f: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:35 master kernel: aufs au_opts_verify:1597:dockerd[3070]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:36 master dockerd[2325]: time="2017-01-06T14:04:36.782641396Z" level=info msg="libcontainerd: new containerd process, pid: 4194"
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.062384572Z" level=info msg="Container failed to stop after sending signal 15 to the process, force killing"
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.063162259Z" level=error msg="fatal task error" error="Cannot stop container whoami2.1.xfrczg5lsdvakvbyf4m8u803l: Cannot kill container abeb1f180a413d511a07e7bebfd50508e19f4ab06b7530429a433bba48aeab33: rpc error: code = 14 desc = grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=xfrczg5lsdvakvbyf4m8u803l
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.062605163Z" level=info msg="Container failed to stop after sending signal 15 to the process, force killing"
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.064055161Z" level=error msg="fatal task error" error="Cannot stop container whoami1.1.6khhwdva9fdeli0bv5i7mszks: Cannot kill container 87bdf0aa25c2c76ce100d8d84b3e5468f1d30147e096862c6df93f1af9b7ab6a: rpc error: code = 14 desc = grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=6khhwdva9fdeli0bv5i7mszks
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.062795503Z" level=info msg="Container failed to stop after sending signal 15 to the process, force killing"
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.065397334Z" level=error msg="fatal task error" error="Cannot stop container whoami3.1.xvr2y12dccsvk4xkijjbiymxc: Cannot kill container e540988a2b9ec87383dee0d75b84eb3fe5ceaa473113b21fedd78e0b2b2b6739: rpc error: code = 14 desc = grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=xvr2y12dccsvk4xkijjbiymxc
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.062542550Z" level=info msg="Container failed to stop after sending signal 15 to the process, force killing"
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.066000202Z" level=error msg="fatal task error" error="Cannot stop container whoami4.1.q8cqnksjqglq2pmdgsfy01kc8: Cannot kill container ea9548ae8fa99b75710806d820189c30ee4db1bc3ea9e93d5d881275858ee937: rpc error: code = 14 desc = grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=q8cqnksjqglq2pmdgsfy01kc8
Jan 06 14:04:38 master dockerd[2325]: time="2017-01-06T14:04:38.282799892Z" level=info msg="libcontainerd: new containerd process, pid: 4201"
Jan 06 14:04:39 master kernel: aufs au_opts_verify:1597:dockerd[2328]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:39 master kernel: aufs au_opts_verify:1597:dockerd[3070]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:39 master dockerd[2325]: time="2017-01-06T14:04:39.791901521Z" level=info msg="libcontainerd: new containerd process, pid: 4208"
Jan 06 14:04:39 master kernel: aufs au_opts_verify:1597:dockerd[2703]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:39 master kernel: device vethca25035 entered promiscuous mode
Jan 06 14:04:39 master kernel: IPv6: ADDRCONF(NETDEV_UP): vethca25035: link is not ready
Jan 06 14:04:39 master kernel: aufs au_opts_verify:1597:dockerd[2890]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:39 master kernel: device veth94e782b entered promiscuous mode
Jan 06 14:04:39 master kernel: IPv6: ADDRCONF(NETDEV_UP): veth94e782b: link is not ready
Jan 06 14:04:39 master kernel: docker0: port 8(veth94e782b) entered forwarding state
Jan 06 14:04:39 master kernel: docker0: port 8(veth94e782b) entered forwarding state
Jan 06 14:04:39 master kernel: device veth0fb4842 entered promiscuous mode
Jan 06 14:04:39 master kernel: IPv6: ADDRCONF(NETDEV_UP): veth0fb4842: link is not ready
Jan 06 14:04:39 master kernel: docker0: port 9(veth0fb4842) entered forwarding state
Jan 06 14:04:39 master kernel: docker0: port 9(veth0fb4842) entered forwarding state
Jan 06 14:04:39 master kernel: device vethbe422b2 entered promiscuous mode
Jan 06 14:04:39 master kernel: IPv6: ADDRCONF(NETDEV_UP): vethbe422b2: link is not ready
Jan 06 14:04:39 master kernel: docker0: port 10(vethbe422b2) entered forwarding state
Jan 06 14:04:39 master kernel: docker0: port 10(vethbe422b2) entered forwarding state
Jan 06 14:04:39 master systemd-udevd[4218]: Could not generate persistent MAC address for veth0fb4842: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4215]: Could not generate persistent MAC address for veth95eaf4a: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4220]: Could not generate persistent MAC address for veth94e782b: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4216]: Could not generate persistent MAC address for vethca25035: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4221]: Could not generate persistent MAC address for vethb81a411: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4222]: Could not generate persistent MAC address for vethbe422b2: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4217]: Could not generate persistent MAC address for vethddef39d: No such file or directory
Jan 06 14:04:39 master systemd-udevd[4219]: Could not generate persistent MAC address for vetha10beef: No such file or directory
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.051554220Z" level=warning msg="Your kernel does not support swap memory limit."
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.051928748Z" level=warning msg="Your kernel does not support cgroup rt period"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.051950077Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.055854092Z" level=warning msg="Your kernel does not support swap memory limit."
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.055939157Z" level=warning msg="Your kernel does not support cgroup rt period"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.055958511Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.066750538Z" level=warning msg="Your kernel does not support swap memory limit."
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.066950753Z" level=warning msg="Your kernel does not support cgroup rt period"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.067515507Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.086500221Z" level=warning msg="Your kernel does not support swap memory limit."
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.086917388Z" level=warning msg="Your kernel does not support cgroup rt period"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.086944401Z" level=warning msg="Your kernel does not support cgroup rt runtime"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.094550325Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.095581602Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.095972723Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.096196053Z" level=error msg="Create container failed with error: grpc: the connection is unavailable"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.097464778Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.097810869Z" level=error msg="Create container failed with error: grpc: the connection is unavailable"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.098507362Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.098587337Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.098740543Z" level=error msg="Create container failed with error: grpc: the connection is unavailable"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.106880316Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.108310098Z" level=error msg="stream copy error: reading from a closed fifo\ngithub.com/docker/docker/vendor/github.com/tonistiigi/fifo.(*fifo).Read\n\t/go/src/github.com/docker/docker/vendor/github.com/tonistiigi/fifo/fifo.go:142\nbufio.(*Reader).fill\n\t/usr/local/go/src/bufio/bufio.go:97\nbufio.(*Reader).WriteTo\n\t/usr/local/go/src/bufio/bufio.go:472\nio.copyBuffer\n\t/usr/local/go/src/io/io.go:380\nio.Copy\n\t/usr/local/go/src/io/io.go:360\ngithub.com/docker/docker/pkg/pools.Copy\n\t/go/src/github.com/docker/docker/pkg/pools/pools.go:60\ngithub.com/docker/docker/container/stream.(*Config).CopyToPipe.func1.1\n\t/go/src/github.com/docker/docker/container/stream/streams.go:119\nruntime.goexit\n\t/usr/local/go/src/runtime/asm_amd64.s:2086"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.108904636Z" level=error msg="Create container failed with error: grpc: the connection is unavailable"
Jan 06 14:04:40 master kernel: docker0: port 7(vethca25035) entered disabled state
Jan 06 14:04:40 master kernel: device vethca25035 left promiscuous mode
Jan 06 14:04:40 master kernel: docker0: port 7(vethca25035) entered disabled state
Jan 06 14:04:40 master kernel: docker0: port 8(veth94e782b) entered disabled state
Jan 06 14:04:40 master kernel: device veth94e782b left promiscuous mode
Jan 06 14:04:40 master kernel: docker0: port 8(veth94e782b) entered disabled state
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.231982759Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/03d3cb812c38fb1de38fc6ad1a5a4e890328380dcd8d667b48bb790e0eb34e9f: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: docker0: port 10(vethbe422b2) entered disabled state
Jan 06 14:04:40 master kernel: device vethbe422b2 left promiscuous mode
Jan 06 14:04:40 master kernel: docker0: port 10(vethbe422b2) entered disabled state
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.258368533Z" level=error msg="fatal task error" error="starting container failed: grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=qs5lp3c41j2o961xfvrlymp5b
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.280005844Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/4d1b5a107a99003f35f4e1a28e0c325c6226e71dde5e202d8a2ba1fabd572bd3: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: docker0: port 9(veth0fb4842) entered disabled state
Jan 06 14:04:40 master kernel: device veth0fb4842 left promiscuous mode
Jan 06 14:04:40 master kernel: docker0: port 9(veth0fb4842) entered disabled state
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.298495124Z" level=error msg="fatal task error" error="starting container failed: grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=lk7ijv9x6ib2pctrb4urot3od
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.312597101Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/67eb5f4dd293db24e093e2cbc3f466714eb560d2b09d277a11898dd5a234893b: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.329940044Z" level=error msg="fatal task error" error="starting container failed: grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=fltzxwnd0ckmcjjzj3w41u5dz
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.364702792Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/96646e55706458468e98038a20bdb79361b4f577e960f9da520c5249a4552c2e: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.383756760Z" level=error msg="fatal task error" error="starting container failed: grpc: the connection is unavailable" module="node/agent/taskmanager" task.id=es4f5b8ozksypkinwkgwxsesz
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2328]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.589315756Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/ff33819524a98beadf94eeab6c0a5b51043585ec9c85b155eb368b8077e2089d-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2697]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.616932842Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/0bb8803e33d1826bacf5dcf1c5bc14ba0105148ebd661899f8c78a44ca319282-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2350]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.637399447Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/ff33819524a98beadf94eeab6c0a5b51043585ec9c85b155eb368b8077e2089d: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2697]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.659131371Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/cf05563fa11108633ffb385d101556a3606aa2b92d5e4775ba744bfc8994ecc6-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.679718294Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/0bb8803e33d1826bacf5dcf1c5bc14ba0105148ebd661899f8c78a44ca319282: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2697]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2328]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.708595948Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/cf05563fa11108633ffb385d101556a3606aa2b92d5e4775ba744bfc8994ecc6: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2350]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.727179124Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/cdb6e28db3ddf476ffd900192a62093847cdd7f06da6d030388c2f570a380e26-init: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:40 master kernel: aufs au_opts_verify:1597:dockerd[2442]: dirperm1 breaks the protection by the permission bits on the lower branch
Jan 06 14:04:40 master dockerd[2325]: time="2017-01-06T14:04:40.743682819Z" level=warning msg="Couldn't run auplink before unmount /var/lib/docker/aufs/mnt/cdb6e28db3ddf476ffd900192a62093847cdd7f06da6d030388c2f570a380e26: exec: \"auplink\": executable file not found in $PATH"
Jan 06 14:04:41 master dockerd[2325]: time="2017-01-06T14:04:41.289796909Z" level=info msg="libcontainerd: new containerd process, pid: 4345"
Jan 06 14:04:42 master dockerd[2325]: time="2017-01-06T14:04:42.792293893Z" level=info msg="libcontainerd: new containerd process, pid: 4352"