olljanat/overlay-network-and-static-ips-for-docker-containers.md

## overlay-network-and-static-ips-for-docker-containers.md

      
    Raw
  

              overlay-network-and-static-ips-for-docker-containers.md
            
          
    I don't expect moby/moby#24170 to be solved by Docker team on near future so this gist is about looking for least ugly workarounds which are available today and on near future.
What is possible on Docker 19.03

Network

It is possible to create overlay network with use user specified subnet.
On this example I create network which is size is two C -class networks ( IP range 10.0.0.0 - 10.0.1.255 )
and force Docker to use IPs from second part of it ( 10.0.1.0 - 10.0.1.255 ).
That way I can make sure that IPs 10.0.0.2 - 10.0.0.254 can be specified to containers and they do not collide with IPs which Docker engine assign for service(s)/container(s).
docker network create --driver overlay --subnet 10.0.0.0/23 --ip-range 10.0.1.0/24 --gateway 10.0.0.1 --attachable test
Containers

By default containers IP addresses are controlled by Docker engine and those cannot be changed inside of container.
Luckily that behavior can be overridden by adding NET_ADMIN capability for container.
Custom Docker image

So first we need create custom docker image which is able set IP for container when it starts.
That we can do example with Dockerfile like this:
FROM ubuntu
RUN apt-get update \
        && apt-get install -y net-tools iputils-ping
COPY /start.sh /
ENTRYPOINT /start.sh

and start script like this:
#!/bin/bash
if [[ -f "${STATIC_IP}" ]]; then
        echo "Using default IP from Docker"
else
        echo "Found static IP: ${STATIC_IP} using it"
        ifconfig eth0 ${STATIC_IP} netmask 255.255.255.0 up
fi
sleep infinity
Then just build that with command:
docker build . -t static-ip
Starting container

Then we can start two containers with commands like these which connects then to overlay network and changes IP addresses inside of the to static ones.
docker run --name test1 --detach=true --rm --cap-add=NET_ADMIN --network=test -e STATIC_IP=10.0.0.11 static-ip
docker run --name test2 --detach=true --rm --cap-add=NET_ADMIN --network=test -e STATIC_IP=10.0.0.12 static-ip
What will be possible on next version

Above solution solves two issues:

It sets static IPs for containers.
It allow containers communicate between Docker nodes using overlay network.

However those containers are not swarm services which means that you lose some control to them.
Luckily I have already implemented docker/cli#1940 and I'm expecting that it will get merged and released as part of next Docker version.
That makes possible to use swarm stack like this:
version: "3.7"
services:
  test1:
    image: static-ip
    environment:
      STATIC_IP: 10.0.0.11
    networks:
      - test
    cap_add:
      - NET_ADMIN
  test2:
    image: static-ip
    environment:
      STATIC_IP: 10.0.0.12
    networks:
      - test
    cap_add:
      - NET_ADMIN

networks:
  test:
   external: true
   name: test
and deploy it with command:
docker stack deploy -c static-ip.yml static-ip