Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
How Docker Swarm handles rescheduling of a container with and without an overlay network
# with overlay network
$ docker network create backend
7496954c6896b2c141902e7c17955289e3f49a0edf51ef0c1ca79dd7f40200a1
$ docker run -d --name redis --net backend -p "6379:6379" -e reschedule:on-node-failure redis
5ecdae84a93ced4406ba2b46e8709069c8651e1a59558a479521d9eb4c00b12e
$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
5ecdae84a93c redis "docker-entrypoint.sh" 11 seconds ago Up 10 seconds 192.168.99.102:6379->6379/tcp swarm-agent-n1/redis
$ redis-cli -h $(docker port redis 6379 | cut -d ':' -f 1) set redis:on:carina true
OK
# kill the node redis is running on (swarm-agent-n1 in this case) and wait 2-3 minutes
$ docker logs swarm-agent-master
...
time="2016-05-12T14:47:17Z" level=info msg="Removed Engine swarm-agent-n1"
time="2016-05-12T14:47:17Z" level=info msg="Rescheduled container 5ecdae84a93ced4406ba2b46e8709069c8651e1a59558a479521d9eb4c00b12e from swarm-agent-n1 to swarm-agent-n2 as 7a403ca8484b696e359ceec15d7ce8161f4bd7d4ada756290eb989caaf3b2ad3"
time="2016-05-12T14:47:17Z" level=info msg="Container 5ecdae84a93ced4406ba2b46e8709069c8651e1a59558a479521d9eb4c00b12e was running, starting container 7a403ca8484b696e359ceec15d7ce8161f4bd7d4ada756290eb989caaf3b2ad3"
time="2016-05-12T14:47:17Z" level=error msg="Failed to start rescheduled container 7a403ca8484b696e359ceec15d7ce8161f4bd7d4ada756290eb989caaf3b2ad3: Error response from daemon: service endpoint with name redis already exists"
$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
7a403ca8484b redis "docker-entrypoint.sh" About a minute ago Created swarm-agent-n2/redis
$ redis-cli -h $(docker port redis 6379 | cut -d ':' -f 1) set redis:on:carina true
Error: No public port '6379/tcp' published for redis
Could not connect to Redis at set:6379: nodename nor servname provided, or not known
# without overlay network
$ docker run -d --name redis -p "6379:6379" -e reschedule:on-node-failure redis
3789a353dfad3539de4d1e8c3a9b0614b75deef6f3b0806b7926f29e19973518
$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3789a353dfad redis "docker-entrypoint.sh" 21 seconds ago Up 20 seconds 192.168.99.102:6379->6379/tcp swarm-agent-n1/redis
$ redis-cli -h $(docker port redis 6379 | cut -d ':' -f 1) set redis:on:carina true
OK
# kill the node redis is running on (swarm-agent-n1 in this case) and wait 2-3 minutes
$ docker logs swarm-agent-master
...
time="2016-05-12T14:56:10Z" level=info msg="Removed Engine swarm-agent-n1"
time="2016-05-12T14:56:10Z" level=info msg="Rescheduled container 3789a353dfad3539de4d1e8c3a9b0614b75deef6f3b0806b7926f29e19973518 from swarm-agent-n1 to swarm-agent-n2 as f1c889ac4e33ee65b211a0bcfe794c08aa153e61866abb205932dfa0c645db1b"
time="2016-05-12T14:56:10Z" level=info msg="Container 3789a353dfad3539de4d1e8c3a9b0614b75deef6f3b0806b7926f29e19973518 was running, starting container f1c889ac4e33ee65b211a0bcfe794c08aa153e61866abb205932dfa0c645db1b"
$ docker ps -l
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f1c889ac4e33 redis "docker-entrypoint.sh" 50 seconds ago Up 49 seconds 192.168.99.103:6379->6379/tcp swarm-agent-n2/redis
$ redis-cli -h $(docker port redis 6379 | cut -d ':' -f 1) get redis:on:carina
(nil)
# result is (nil) because data got turfed but at least we can communicate with redis
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.