Skip to content

Instantly share code, notes, and snippets.

@jesugmz

jesugmz/multi-cloud-docker-swarm.md

Last active January 2, 2024 18:40
Show Gist options
  • Star 16 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save jesugmz/2bbf41f947c6f5fb33fbb3aa2b84b2b8 to your computer and use it in GitHub Desktop.
Save jesugmz/2bbf41f947c6f5fb33fbb3aa2b84b2b8 to your computer and use it in GitHub Desktop.
Download ZIP
Create a simple multi cloud Docker cluster using Docker Swarm, Docker Machine and the three top cloud providers nowadays - Google Compute Engine, Microsoft Azure and AWS
Raw
multi-cloud-docker-swarm.md

Simple multi cloud Docker cluster using Docker Swarm

This guide explains how to create a simple multi cloud Docker cluster using Docker Swarm, Docker Machine and the three top cloud providers nowadays - Google Compute Engine, Microsoft Azure and AWS.

Prerequisites

This guide assumes you have a Linux host with Docker CE installed. If you are using Docker for Mac or Docker for Windows you can avoid the Docker Machine set up since it comes included.

Install Docker Machine

The nodes provisioning will be done using Docker Machine. Follow the official guide to install it.

Nodes provisioning

Create instance in GCE - Google Compute Engine

It assumes you have already an GCE account with at least one project.

Install Google Cloud SDK

Follow https://cloud.google.com/sdk/docs/quickstarts.

Configure credentials

  1. Go to https://console.cloud.google.com/apis/credentials/serviceaccountkey
  2. In Service Account choose Compute Engine...
  3. In Key type keep JSON
  4. Click in Create to download

After download the JSON file, put it in a location and export it as follow:

export GOOGLE_APPLICATION_CREDENTIALS=<CREDENTIALS_ABSOLUTE_PATH>

Create the instance

Assuming we want to create a Debian 8 based instance:

docker-machine create --driver google \
    --google-project <PROJECT_ID> \
    --google-machine-type f1-micro \
    --google-zone europe-west1-b \
    --google-machine-image https://www.googleapis.com/compute/v1/projects/debian-cloud/global/images/debian-8-jessie-v20180510 \
    gce1

Docker Machine will create the remote instance in GCE and provision it with Docker Engine.

After the execution docker-machine ls will show the new remote host:

NAME   ACTIVE   DRIVER   STATE     URL                        SWARM   DOCKER        ERRORS
gce1   -        google   Running   tcp://35.189.XX.XXX:2376           v18.05.0-ce

Notes

  • GCE allows to sign up with prepaid credit card
  • $300 free credit for 12 first months
  • f1-micro single instance is free forever
  • Docker Machine driver reference for GCE https://docs.docker.com/machine/drivers/gce/
  • The Docker Machine driver does not work with the configuration and authentication created with the GCE SDK CLI tool
  • To check the list of images gcloud compute images list --uri
  • To check the list of zones gcloud compute zones list
  • The full list of zones with hardware details can be found here https://cloud.google.com/compute/docs/regions-zones/
  • The zone europe-west1-b has the most powerful CPU in Europe nowadays
  • Complex UI but intuitive CLI SDK

Create instance in Microsoft Azure

Install Azure CLI

Follow https://docs.microsoft.com/en-us/cli/azure/install-azure-cli?view=azure-cli-latest.

Create the instance

Assuming we want to create a Debian 8 based instance:

docker-machine create --driver azure \
    --azure-subscription-id <SUBSCRIPTION_ID> \
    --azure-size Standard_B1s \
    --azure-location westeurope \
    --azure-image credativ:Debian:8:latest \
    ma1

After the execution docker-machine ls will show the new remote host:

NAME   ACTIVE   DRIVER   STATE     URL                        SWARM   DOCKER        ERRORS
gce1   -        google   Running   tcp://35.189.XXX.XX:2376           v18.05.0-ce   
ma1    -        azure    Running   tcp://23.97.XXX.XXX:2376           v18.05.0-ce

Notes

Create instance in AWS - Amazon Web Services

Install AWS CLI

Follow https://docs.aws.amazon.com/cli/latest/userguide/installing.html.

Configure credentials

  1. Go to https://console.aws.amazon.com/iam/home?#/security_credential and create a new Access keys if you do not have one already
  2. Execute aws configure and fill the steps

Create the instance

Assuming we want to create a Debian 8 based instance:

docker-machine create --driver amazonec2 \
    --amazonec2-instance-type t2.micro \
    --amazonec2-region eu-central-1 \
    --amazonec2-ami ami-5900cc36 \
    --amazonec2-device-name /dev/xvda \
    --amazonec2-ssh-user admin \
    aws1

After the execution docker-machine ls will show the new remote host:

NAME   ACTIVE   DRIVER      STATE     URL                        SWARM   DOCKER        ERRORS
aws1   -        amazonec2   Running   tcp://35.157.XX.XX:2376            v18.05.0-ce   
gce1   -        google      Running   tcp://35.189.XXX.XX:2376           v18.05.0-ce   
ma1    -        azure       Running   tcp://23.97.XXX.XXX:2376           v18.05.0-ce

Notes

  • t2.micro instance is 12 month free (750 hours/month)
  • Docker Machine driver reference for AWS https://docs.docker.com/machine/drivers/aws/
  • To check the list of regions aws ec2 describe-regions
  • To check the official list of Debian Jessie images aws ec2 describe-images --owners 379101102735 --filters "Name=name,Values=debian-jessie-*"
  • More info about official Debian Jessie AMIs https://wiki.debian.org/Cloud/AmazonEC2Image/Jessie
  • High number of AMIs and its custom configuration makes the process of choose and create instances complex

Creating the cluster

After the nodes provisioning we can proceed with the creation of the Docker Swarm.

Create the manager

Lets use the GCE instance as manager:

  1. SSH to the GCE instance docker-machine ssh gce1
  2. Make this node as manager in the Docker Swarm sudo docker swarm init --advertise-addr <THIS_HOST_IP>

The output of the last command should be something like this:

Swarm initialized: current node (jycadm63k7m5162bxvkqlgsem) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-2aon4rvr50khj9cb7w1t80610gxxpl9v8gkc7n7qf8yxj03prc-1ozrrrs676zgqb0vcvekkiyqk 35.189.xxx.xx:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

By default GCE only allows very specific incoming traffic so we have to proceed configuring the firewall as follow:

Allow cluster management communications

gcloud compute firewall-rules create docker-swarm-manager \
    --action allow \
    --source-ranges 0.0.0.0/0 \
    --rules tcp:2377

Allow communication among nodes

gcloud compute firewall-rules create docker-swarm-discovery \
    --action allow \
    --source-ranges 0.0.0.0/0 \
    --rules tcp:7946,udp:7946

Allow overlay network traffic

gcloud compute firewall-rules create docker-swarm-network \  
    --action allow \
    --source-ranges 0.0.0.0/0 \
    --rules udp:4789

More information related with the use of the firewall can be found in Using Firewall Rules.

Now we can exit from this host.

If you need to remain this output, execute docker swarm join-token worker from this manager host any time.

Add nodes to the manager

Azure

  1. SSH to the Microsoft Azure instance docker-machine ssh ma1
  2. Lets join this node to the Docker Swarm sudo docker swarm join --token SWMTKN-1-2aon4rvr50khj9cb7w1t80610gxxpl9v8gkc7n7qf8yxj03prc-1ozrrrs676zgqb0vcvekkiyqk 35.189.xxx.xx:2377

The output should be:

This node joined a swarm as a worker.

Exit from this host.

Now, from the manager host, if we execute docker node ls we should see something like this:

NAME   ACTIVE   DRIVER   STATE     URL                        SWARM   DOCKER        ERRORS
gce1   -        google   Running   tcp://35.189.xxx.xx:2376           v18.05.0-ce   
ma1    -        azure    Running   tcp://23.97.xxx.xxx:2376           v18.05.0-ce

Proceed allowing needed traffic in the firewall:

Allow communication among nodes

az network nsg rule create \
    --resource-group docker-machine \
    --nsg-name ma1-firewall \
    --name dockerSwarmDiscovery \
    --priority 301 \
    --access Allow \
    --protocol '*' \
    --destination-port-range 7946

Allow overlay network traffic

az network nsg rule create \
    --resource-group docker-machine \
    --nsg-name ma1-firewall \
    --name dockerSwarmNetwork \
    --priority 302 \
    --access Allow \
    --protocol udp \
    --destination-port-range 4789

More information about firewall management using az network nsg rule CLI command.

AWS

Same operation for the AWS instance:

  1. docker-machine ssh aws1
  2. sudo docker swarm join --token SWMTKN-1-2aon4rvr50khj9cb7w1t80610gxxpl9v8gkc7n7qf8yxj03prc-1ozrrrs676zgqb0vcvekkiyqk 35.189.XXX.XX:2377

And exit from this host.

Once again, from the manager host, docker node ls should show something like this:

ID                            HOSTNAME            STATUS              AVAILABILITY        MANAGER STATUS      ENGINE VERSION
iir58ycxlfgxldfmyc4ztx8mf     aws1                Ready               Active                                  18.05.0-ce
jycadm63k7m5162bxvkqlfsem *   gce1                Ready               Active              Leader              18.05.0-ce
vur0qe657al0591uhyab4r6f4     ma1                 Ready               Active                                  18.05.0-ce

Proceed allowing needed traffic in the firewall:

Allow communication among nodes

aws ec2 authorize-security-group-ingress \
    --group-name docker-machine \
    --cidr 0.0.0.0/0 \
    --port 7946 \
    --protocol tcp

aws ec2 authorize-security-group-ingress \
    --group-name docker-machine \
    --cidr 0.0.0.0/0 \
    --port 7946 \
    --protocol udp

Allow overlay network traffic

aws ec2 authorize-security-group-ingress \
    --group-name docker-machine \
    --cidr 0.0.0.0/0 \
    --port 4789 \
    --protocol udp

More information related with the use of the firewall can be found in authorize-security-group-ingress.

Deploy services in the cluster

Kernel IP forwarding comes disabled by default in GCE, to enable it:

gcloud compute ssh gce1 \
    --command 'sudo sed -i 's/#net.ipv4.ip_forward=1/net.ipv4.ip_forward=1/g' /etc/sysctl.conf; sudo sysctl -p'
  1. SSH to the manager instance - GCE in our example docker-machine ssh gce1
  2. Create the service:
sudo docker service create \
    --replicas 3 \
    --name webserver \
    --publish published=80,target=80 \
    nginx
  1. Open port 80 in each host using its CLI:

GCE

gcloud compute firewall-rules create docker-swarm-webserver \
    --action allow \
    --source-ranges 0.0.0.0/0 \
    --rules tcp:80

Azure

az network nsg rule create \
    --resource-group docker-machine \
    --nsg-name ma1-firewall \
    --name dockerSwarmWebserver \
    --priority 1000 \
    --access Allow \
    --protocol tcp \
    --destination-port-range 80

AWS

aws ec2 authorize-security-group-ingress \
    --group-name docker-machine \
    --cidr 0.0.0.0/0 \
    --protocol tcp \
    --port 80

Now we have three Nginx instances running in our cluster where three can be increased or decreased on demand using docker service update.

Thanks to the routing mesh of Docker Swarm the service can be reach from any IP that belongs to a node in the cluster.

alt text

To get the all the powerful features of Docker Swarm is ideal to go with an immutable infrastructure.

Guide based on the Docker Swarm tutorial.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment