The world is constantly being attracted towards containers and its orchestration especially in large and dynamic environments which has also led to the rising of various tools and platforms in trend.
As we all know that Docker is a major player in the world of containers and therefore in this blog, we will be exploring what it offers in terms of orchestration.
Yes, we will be looking at Docker-Swarm and some of its key components !!!
Also here I am assuming that the reader is prior aware of the basic docker concepts like container, image, etc.
Swarm is a fully integrated and built-in container orchestration tool provided by Docker. It is a set of nodes i.e Docker hosts which are running in swarm mode with at least one master node and several worker nodes that can be virtual or physical machines.
It allows the user to manage multiple containers deployed across multiple host machines also while in swarm mode, you can still run standalone containers on any of the Docker hosts.
But when a container is running in service mode, swarm takes over and manages the lifecycle of the service and constantly maintains the desired number of replicas, as specified in the definition.
Now as we have covered a bit about Docker-Swarm, let's look at its components now.
The major components of a swarm cluster includes nodes ( worker & manager ), services, docker client.
A node is a machine that joins the swarm cluster and each of the node contains an instance of docker engine. Each node of a docker swarm is a docker daemon and all of them interact with docker API over HTTP.
There are 2 types of nodes in the swarm cluster :
-
Manager Node
- The main function of the manager node is to assign tasks to worker nodes in the swarm and keeps track of the services running.
- It is responsible for all orchestration and container management tasks required to maintain the desired state of the cluster.
- There can be multiple manager nodes and based on the Raft consensus algorithm one of them gets assigned as the "leader node".
-
Worker Node
- These nodes receive and execute tasks distributed by the manager node in the swarm.
- Each worker node runs an agent that reports back to the master node about the state of the tasks assigned.
- By default manager nodes also run services as worker nodes which can be configured to run as manager only (if needed).
Now to execute something or to deploy an application on swarm cluster it should be properly defined as service. It specifies which container image to use and which commands to execute inside running containers and other stuffs like which port to use, update policy, etc.
Again there are 2 types of services:
- Global Service, in which the swarm runs one task for the service on every available node in the cluster.
- Replicated Service, in which the swarm manager distributes a specific number of replica tasks among the nodes based upon the scale you set in the desired state.
Tasks is the most basic and smallest unit of scheduling within a swarm. Manager nodes assign tasks to worker nodes from the service in action, and after this assignment, the task cannot be moved to another worker.
A task is a uni-directional and can go through a series of state i.e assigned, prepared, running, etc.
If the task fails, the manager will assign a new version of that task to another available node in the swarm.
The cluster will have 3 worker node and a master node and we will be using docker-machine and virtualbox to get our virtual machines ready with docker-engine installed.
To check the status of docker-machine on your system
$ docker-machine -v
docker-machine version 0.16.0, build 702c267f
Now bring one machine at a time (might take some time).
$ docker-machine create --driver virtualbox manager
$ docker-machine create --driver virtualbox worker1
$ docker-machine create --driver virtualbox worker2
$ docker-machine create --driver virtualbox worker3
After that list the machines.
$ docker-machine ls
NAME ACTIVE DRIVER STATE URL SWARM DOCKER ERRORS
manager - virtualbox Running tcp://192.168.99.100:2376 v19.03.5
worker1 - virtualbox Running tcp://192.168.99.101:2376 v19.03.5
worker2 - virtualbox Running tcp://192.168.99.102:2376 v19.03.5
worker3 - virtualbox Running tcp://192.168.99.103:2376 v19.03.5
SSH into the manager virtual machine and run the following commands to join as manager and worker individually on each node.
$ sudo docker-machine ssh manager
docker@manager:~$ docker swarm init --advertise-addr 192.168.99.100
You must get a similar output
Swarm initialized: current node (q1hry94rx04jqk7r5hwiuogbv) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-2uir91nxhwob7m9mlyn7fm80meofeia9w2hsx59853cxe1p7a0-bo75x86zw7gv3f2wev80fdm4g 192.168.99.100:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
$ docker swarm join-token manager
To add a manager to this swarm, run the following command:
docker swarm join --token SWMTKN-1-2uir91nxhwob7m9mlyn7fm80meofeia9w2hsx59853cxe1p7a0-c25v8ib80i4udjnrhush4sb3x 192.168.99.100:2377
Now SSH into each worker machine and run below command to join as worker
$ sudo docker-machine ssh worker1
docker@worker1:~$docker swarm join --token SWMTKN-1-2uir91nxhwob7m9mlyn7fm80meofeia9w2hsx59853cxe1p7a0-bo75x86zw7gv3f2wev80fdm4g 192.168.99.100:2377
This node joined a swarm as a worker.
Follow the same steps for other workers as well.
Now verify that all nodes have joined the swarm cluster. For that go to the master node and run following command:
docker@manager:~$ docker node ls
ID HOSTNAME STATUS AVAILABILITY MANAGER STATUS ENGINE VERSION
q1hry94rx04jqk7r5hwiuogbv * manager Ready Active Leader 19.03.5
w7j9tig7elf6cgafxu6kv8flg worker1 Ready Active 19.03.5
j92xu0td5lvxex1rk11w3bkmq worker2 Ready Active 19.03.5
As you can see that all the nodes have joined and also note the manager status to see which master node is elected as leader(in case if you have multiple master nodes).
Now just to get an idea how services work in swarm cluster and for the sake of simplicity we will be deploying an nginx image .
docker@manager:~$ docker service create --name my-nginx-app --replicas 3 -p 80:80 surajsaini95/my-nginx-app:1.0
o19z75qzv99t5ytib1bldfxju
overall progress: 3 out of 3 tasks
1/3: running [==================================================>]
2/3: running [==================================================>]
3/3: running [==================================================>]
verify: Service converged
Also you can verify by listing the services on manager and it will also show that which node is currently executing the task dispatched.
docker@manager:~$ docker service ls
ID NAME MODE REPLICAS IMAGE PORTS
o19z75qzv99t my-nginx-app replicated 3/3 surajsaini95/my-nginx-app:1.0 *:80->80/tcp
d
docker@manager:~$ docker service ps my-nginx-app
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
ejxp2a4ajd9v my-nginx-app.1 surajsaini95/my-nginx-app:1.0 worker3 Running Running 2 minutes ago
p1caegrzcmqs my-nginx-app.2 surajsaini95/my-nginx-app:1.0 manager Running Running 2 minutes ago
sgja8xcp6gp8 my-nginx-app.3 surajsaini95/my-nginx-app:1.0 worker2 Running Running 2 minutes ago
Now for that go to your browser and hit any of the IP of the nodes which have joined the swarm cluster and it will present you the modified index page of Nginx. Note here that even the node which is not executing the task currently (worker1 in my case )will also present you the same output . This is the magic of service discovery provided by swarm.
To scale application up and down you can use the following command:
docker@manager:~$ docker service scale my-nginx-app=5
my-nginx-app scaled to 5
overall progress: 5 out of 5 tasks
1/5: running [==================================================>]
2/5: running [==================================================>]
3/5: running [==================================================>]
4/5: running [==================================================>]
5/5: running [==================================================>]
verify: Service converged
Now verify it again
docker@manager:~$ docker service ps my-nginx-app
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
ejxp2a4ajd9v my-nginx-app.1 surajsaini95/my-nginx-app:1.0 worker3 Running Running about an hour ago
p1caegrzcmqs my-nginx-app.2 surajsaini95/my-nginx-app:1.0 manager Running Running about an hour ago
sgja8xcp6gp8 my-nginx-app.3 surajsaini95/my-nginx-app:1.0 worker2 Running Running about an hour ago
uso3ywdnycsd my-nginx-app.4 surajsaini95/my-nginx-app:1.0 worker1 Running Running 10 seconds ago
od87wesfyx31 my-nginx-app.5 surajsaini95/my-nginx-app:1.0 worker1 Running Running 10 seconds ago
To smoothly update you app use the following command
docker@manager:~$ docker service update my-nginx-app --image surajsaini95/my-nginx-app:2.0
my-nginx-app
overall progress: 5 out of 5 tasks
1/5: running [==================================================>]
2/5: running [==================================================>]
3/5: running [==================================================>]
4/5: running [==================================================>]
5/5: running [==================================================>]
verify: Service converged
Now swarm will shut down the old container one at a time and run a new container with the updated image. This ensures smooth rollout of updated with zero downtime.
Run the following command to see this process:
docker@manager:~$ docker service ps my-nginx-app
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
4xrxwy70ksgl my-nginx-app.1 surajsaini95/my-nginx-app:2.0 worker3 Running Running about a minute ago
ejxp2a4ajd9v \_ my-nginx-app.1 surajsaini95/my-nginx-app:1.0 worker3 Shutdown Shutdown about a minute ago
p42vctuv3twl my-nginx-app.2 surajsaini95/my-nginx-app:2.0 manager Running Running about a minute ago
p1caegrzcmqs \_ my-nginx-app.2 surajsaini95/my-nginx-app:1.0 manager Shutdown Shutdown about a minute ago
yhnfn7yefuzq my-nginx-app.3 surajsaini95/my-nginx-app:2.0 worker2 Running Running 53 seconds ago
sgja8xcp6gp8 \_ my-nginx-app.3 surajsaini95/my-nginx-app:1.0 worker2 Shutdown Shutdown about a minute ago
0hm1fb13ghw1 my-nginx-app.4 surajsaini95/my-nginx-app:2.0 worker1 Running Running 42 seconds ago
uso3ywdnycsd \_ my-nginx-app.4 surajsaini95/my-nginx-app:1.0 worker1 Shutdown Shutdown 50 seconds ago
1g7ca3qdauam my-nginx-app.5 surajsaini95/my-nginx-app:2.0 manager Running Running about a minute ago
od87wesfyx31 \_ my-nginx-app.5 surajsaini95/my-nginx-app:1.0 worker1 Shutdown Shutdown about a minute ago
To shut down a particular node use the following command
docker@manager:~$docker node update --availability drain worker3
To remove a particular service from swarm cluster
docker@manager:~$docker service rm my-nginx-app
docker@worker1:~$docker swarm leave
Now perform a final clean-up of the VMs which we have created for nodes.
$ docker-machine stop worker3
$ docker-machine stop worker2
$ docker-machine stop worker1
$ docker-machine stop master
We have seen what is docker-swarm and how it works, now let's see why we should use it and what are the benefits it offers :
- Scaling through desired state reconciliation
- Multi-host networking
- Service discovery through embedded DNS
- Load balancing
- And many more...
There are also other tools and platform available for the container orchestration purpose which are being used in the production environment as well. Some of them are :
- Kubernetes
- CoreOS Fleet
- Mesosphere Marathon
- And many more...