Skip to content

Instantly share code, notes, and snippets.

@versionsix

versionsix/docker_l3_routed_ipvlan.md

Created April 10, 2021 05:26
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save versionsix/a5053a2dce23a4adb09952e2d07531c6 to your computer and use it in GitHub Desktop.
Save versionsix/a5053a2dce23a4adb09952e2d07531c6 to your computer and use it in GitHub Desktop.
Download ZIP
Docker ipvlan use a routed subnet.
Raw
docker_l3_routed_ipvlan.md 
❯ multipass launch \
  -m 8G \
  -c 4 \
  -d 30G \
  --name dockrr \
    focal
Launched: dockrr
❯ DOCKRR_IPV4=$(multipass info dockrr --format json | jq -r '.info.dockrr.ipv4[0]' | tee /dev/stderr)
10.196.62.94

❯ sudo ip r add 192.168.210.0/24 via $DOCKRR_IPV4

❯ sudo ip link add name dummy1 type dummy

❯ sudo ip link set dev dummy1 up

❯ sudo ip addr add 192.168.168.168/32 dev dummy1

❯ ip -4 -c r get 192.168.210.0
192.168.210.0 via 10.196.62.94 dev mpqemubr0 src 10.196.62.1 uid 1000
    cache

❯ ip -c -4 addr show dev mpqemubr0
9: mpqemubr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.196.62.1/24 brd 10.196.62.255 scope global mpqemubr0
       valid_lft forever preferred_lft forever
       
❯ ip -c -4 addr show dev dummy1
12: dummy1: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 198.51.100.100/24 scope global dummy1
       valid_lft forever preferred_lft forever
    inet 192.168.168.168/32 scope global dummy1
       valid_lft forever preferred_lft forever

❯ multipass exec dockrr -- sh -c "curl -fsSLo- https://get.docker.com | bash -s"
# Executing docker install script, commit: 7cae5f8b0decc17d6571f9f52eb840fbc13b2737
+ sudo -E sh -c 'apt-get update -qq >/dev/null'
+ sudo -E sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y -qq apt-transport-https ca-certificates curl >/dev/null'
+ sudo -E sh -c 'curl -fsSL "https://download.docker.com/linux/ubuntu/gpg" | apt-key add -qq - >/dev/null'
Warning: apt-key output should not be parsed (stdout is not a terminal)
+ sudo -E sh -c 'echo "deb [arch=amd64] https://download.docker.com/linux/ubuntu focal stable" > /etc/apt/sources.list.d/docker.list'
+ sudo -E sh -c 'apt-get update -qq >/dev/null'
+ '[' -n '' ']'
+ sudo -E sh -c 'apt-get install -y -qq --no-install-recommends docker-ce >/dev/null'
+ '[' -n 1 ']'
+ sudo -E sh -c 'DEBIAN_FRONTEND=noninteractive apt-get install -y -qq docker-ce-rootless-extras >/dev/null'
+ sudo -E sh -c 'docker version'
Client: Docker Engine - Community
 Version:           20.10.5
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        55c4c88
 Built:             Tue Mar  2 20:18:20 2021
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.5
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       363e9a8
  Built:            Tue Mar  2 20:16:15 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0
<omit>
multipass exec dockrr -- /bin/bash <<EOF
cat <<EON | sudo tee /etc/sysctl.d/20-net-open-for-debug.conf
net.ipv4.ip_forward=1
net.ipv4.conf.all.forwarding=1
net.ipv4.conf.all.log_martians = 1
net.ipv6.conf.all.forwarding=1
net.ipv4.conf.default.log_martians=1
net.ipv4.conf.all.rp_filter=0
net.ipv4.conf.default.rp_filter=0
net.ipv4.conf.default.accept_local=1
net.ipv4.conf.all.accept_local=1
net.core.default_qdisc=fq
net.ipv4.tcp_congestion_control=bbr
EON
sudo sysctl -p /etc/sysctl.d/20-net-open-for-debug.conf
sudo systemctl restart procps
sudo usermod -aG docker $USER
newgrp docker
docker info

sudo apt update && sudo apt install -y -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" -o Dpkg::Progress-Fancy="0" jq

DOCKRR_DG_IF=$(ip --json r show default | jq -r '.[0].dev' | tee /dev/stderr)
# docker network create -d ipvlan --subnet 192.168.210.0/24 --gateway=192.168.210.1 -o ipvlan_mode=l2 -o parent=$DOCKRR_DG_IF ipvlan210
docker network create -d ipvlan --subnet 192.168.210.0/24 -o ipvlan_mode=l3 -o parent=$DOCKRR_DG_IF ipvlan210
docker network ls -f driver=ipvlan
docker network inspect ipvlan210

mkdir -p ~/ubuntunettools
cat <<EON > ~/ubuntunettools/Dockerfile
FROM ubuntu:20.04

RUN echo "**** install runtime packages ****" && \
  apt -y \
    -o Dpkg::Progress-Fancy="0" \
    -o Debug::Acquire::http=false \
    -o Acquire::http::Timeout="10" \
    -o Acquire::ftp::Timeout="10" \
      update && \
  apt -y \
    -o Dpkg::Progress-Fancy="0" \
    -o Debug::Acquire::http=false \
    -o Acquire::http::Timeout="10" \
    -o Acquire::ftp::Timeout="10" \
    -o Dpkg::Options::="--force-confdef" \
    -o Dpkg::Options::="--force-confold" \
    --no-install-recommends \
      install \
        bind9-dnsutils \
        coreutils \
        curl \
        iproute2 \
        iputils-arping \
        iputils-ping \
        jq \
        less \
        mtr \
        nano \
        procps \
        psmisc \
        tcpdump \
        telnet \
        traceroute \
          && \
    apt -y autoremove && \
    rm -Rf /var/lib/apt/lists/* && \
    rm -Rf /usr/share/{doc,man} && \
    apt clean
EON
pushd ~/ubuntunettools
docker build -t ubuntunettools .
popd
EOF

Okay so now the vm with ducker has it's net. Let's try some vm stuff. This implies we enter the vm.

- multipass shell dockrr

ubuntu@dockrr:~$ docker run --rm -t -i --network=ipvlan210 ubuntunettools bash -c "ip -c -4 addr show dev eth0"
13: eth0@if2: <BROADCAST,MULTICAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default
    inet 192.168.210.2/24 brd 192.168.210.255 scope global eth0
       valid_lft forever preferred_lft forever

ubuntu@dockrr:~$ docker run --rm -t -i --network=ipvlan210 ubuntunettools bash -c "ip -c -4 r show"
default dev eth0 scope link
192.168.210.0/24 dev eth0 proto kernel scope link src 192.168.210.2

ubuntu@dockrr:~$ docker run --rm -t -i --network=ipvlan210 ubuntunettools bash -c "ping -c2 192.168.168.168"
PING 192.168.168.168 (192.168.168.168) 56(84) bytes of data.
64 bytes from 192.168.168.168: icmp_seq=1 ttl=64 time=0.225 ms
64 bytes from 192.168.168.168: icmp_seq=2 ttl=64 time=0.219 ms

--- 192.168.168.168 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 0.219/0.222/0.225/0.003 ms

On the bare metal host we we see the pings.

❯ sudo tcpdump -nnvvS -i any host 192.168.168.168
tcpdump: listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
07:22:17.873923 IP (tos 0x0, ttl 64, id 46426, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.210.2 > 192.168.168.168: ICMP echo request, id 1, seq 1, length 64
07:22:17.873923 IP (tos 0x0, ttl 64, id 46426, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.210.2 > 192.168.168.168: ICMP echo request, id 1, seq 1, length 64
07:22:17.873975 IP (tos 0x0, ttl 64, id 16468, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.168.168 > 192.168.210.2: ICMP echo reply, id 1, seq 1, length 64
07:22:17.873982 IP (tos 0x0, ttl 64, id 16468, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.168.168 > 192.168.210.2: ICMP echo reply, id 1, seq 1, length 64
07:22:18.899889 IP (tos 0x0, ttl 64, id 46548, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.210.2 > 192.168.168.168: ICMP echo request, id 1, seq 2, length 64
07:22:18.899889 IP (tos 0x0, ttl 64, id 46548, offset 0, flags [DF], proto ICMP (1), length 84)
    192.168.210.2 > 192.168.168.168: ICMP echo request, id 1, seq 2, length 64
07:22:18.899934 IP (tos 0x0, ttl 64, id 16707, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.168.168 > 192.168.210.2: ICMP echo reply, id 1, seq 2, length 64
07:22:18.899941 IP (tos 0x0, ttl 64, id 16707, offset 0, flags [none], proto ICMP (1), length 84)
    192.168.168.168 > 192.168.210.2: ICMP echo reply, id 1, seq 2, length 64
^C
8 packets captured
8 packets received by filter
0 packets dropped by kernel

Okay time for bed. Bye!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment