Skip to content

Instantly share code, notes, and snippets.

@dougbtv

dougbtv/README.md

Created August 2, 2019 15:25
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dougbtv/72f8f969e110b1b1087dbedc07fc624b to your computer and use it in GitHub Desktop.
Save dougbtv/72f8f969e110b1b1087dbedc07fc624b to your computer and use it in GitHub Desktop.
Download ZIP
Installing cni-route-override and adding an additional route
Raw
README.md

Installation and usage of cni-route-override

Covers installing & configuring an example cni-route-override instance.

This example simply adds a static route.

Install route-override binaries

Install using the provided daemonset, this will copy the route-override binary to the /opt/cni/bin on each machine in your Kubernetes cluster.

git clone https://github.com/redhat-nfvpe/cni-route-override.git && cd cni-route-override/
kubectl create -f deployments/daemonset-install.yaml

Create a CNI configuration that chains the route-override plugin

cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: example-route-override
spec:
  config: '{
      "cniVersion": "0.3.0",
      "name": "example-route-override",
      "plugins": [
        {
          "type": "macvlan",
          "master": "eth0",
          "mode": "bridge",
          "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16"
          }
        },
        {
          "type": "route-override",
          "addroutes": [ {
              "dst": "192.168.0.0/24",
              "gw": "10.88.0.254"
          } ],
          "flushgateway": false,
          "flushroutes": false 
        }
      ]
}'
EOF

Create a pod that references that configuration...

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: route-override-sample
  annotations:
    k8s.v1.cni.cncf.io/networks: example-route-override
spec:
  containers:
  - name: route-override-sample
    command: ["/bin/bash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: dougbtv/centos-network
    securityContext:
      privileged: true
EOF

Verify that the additional route exists...

[centos@kube-netmachine-master cni-route-override]$ kubectl exec -it route-override-sample -- /bin/bash
[root@route-override-sample /]# ip route
default via 10.244.1.1 dev eth0 
10.88.0.0/16 dev net1  proto kernel  scope link  src 10.88.0.3 
10.244.0.0/16 via 10.244.1.1 dev eth0 
10.244.1.0/24 dev eth0  proto kernel  scope link  src 10.244.1.166 
192.168.0.0/24 via 10.88.0.254 dev net1 
[root@route-override-sample /]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if167: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP 
    link/ether 9e:72:6b:95:89:b6 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.244.1.166/24 scope global eth0
       valid_lft forever preferred_lft forever
4: net1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 26:41:90:34:18:89 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.88.0.3/16 scope global net1
       valid_lft forever preferred_lft forever

Additional example options

Courtesy of to this gist by Tomofumi Hayashi!

# Initial Config (without cni-route-override)
[tohayash@tohayash-lab ~]$ cat /etc/cni/net.d/87-podman-bridge.conflist 
{
    "cniVersion": "0.3.1",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }, 
                { "dst": "192.1.1.0/24", "gw": "10.88.0.254" }
            ]
        }
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}
[tohayash@tohayash-lab ~]$ sudo podman run --network=podman -it docker.io/centos/tools:latest bash
[root@61c4fb920ba8 /]# ip route
default via 10.88.0.1 dev eth0 
10.88.0.0/16 dev eth0 proto kernel scope link src 10.88.0.33 
192.1.1.0/24 via 10.88.0.254 dev eth0 

# with cni-route-override (just added, nothing happen ;) 
[tohayash@tohayash-lab ~]$ cat 87-podman-bridge.conflist 
{
    "cniVersion": "0.3.1",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }, 
                { "dst": "192.1.1.0/24", "gw": "10.88.0.254" }
            ]
        }
      },
      {
        "type": "route-overwrite"
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}
[tohayash@tohayash-lab ~]$ sudo podman run --network=podman -it docker.io/centos/tools:latest bash
[root@6b5579af452f /]# ip route
default via 10.88.0.1 dev eth0 
10.88.0.0/16 dev eth0 proto kernel scope link src 10.88.0.34 
192.1.1.0/24 via 10.88.0.254 dev eth0 

# Flush all routes with cni-route-override
[tohayash@tohayash-lab ~]$ cat 87-podman-bridge.conflist 
{
    "cniVersion": "0.3.1",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }, 
                { "dst": "192.1.1.0/24", "gw": "10.88.0.254" }
            ]
        }
      },
      {
        "type": "route-overwrite",
        "flushgateway": false,
        "flushroutes": true
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}
[tohayash@tohayash-lab ~]$ sudo podman run --network=podman -it docker.io/centos/tools:latest bash
[root@272646f92216 /]# ip route
10.88.0.0/16 dev eth0 proto kernel scope link src 10.88.0.35 

Note: in case of flush all routes, we keep interface routes above.

# Flush gateway only
[tohayash@tohayash-lab ~]$ cat /etc/cni/net.d/87-podman-bridge.conflist 
{
    "cniVersion": "0.3.1",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }, 
                { "dst": "192.1.1.0/24", "gw": "10.88.0.254" }
            ]
        }
      },
      {
        "type": "route-overwrite",
        "flushgateway": true,
        "flushroutes": false 
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}
[tohayash@tohayash-lab ~]$ sudo podman run --network=podman -it docker.io/centos/tools:latest bash
[root@48d9f2e3d229 /]# ip route
10.88.0.0/16 dev eth0 proto kernel scope link src 10.88.0.37 
192.1.1.0/24 via 10.88.0.254 dev eth0 

# Add route
[tohayash@tohayash-lab ~]$ cat /etc/cni/net.d/87-podman-bridge.conflist 
{
    "cniVersion": "0.3.1",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }, 
                { "dst": "192.1.1.0/24", "gw": "10.88.0.254" }
            ]
        }
      },
      {
        "type": "route-overwrite",
        "addroutes": [ {
            "dst": "192.168.0.0/24",
            "gw": "10.88.0.254"
        } ],
        "flushgateway": false,
        "flushroutes": false 
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}
[tohayash@tohayash-lab ~]$ sudo podman run --network=podman -it docker.io/centos/tools:latest bash
[root@22969ab6ab24 /]# ip route
default via 10.88.0.1 dev eth0 
10.88.0.0/16 dev eth0 proto kernel scope link src 10.88.0.38 
192.1.1.0/24 via 10.88.0.254 dev eth0 
192.168.0.0/24 via 10.88.0.254 dev eth0 

# Delete route
[tohayash@tohayash-lab ~]$ cat /etc/cni/net.d/87-podman-bridge.conflist 
{
    "cniVersion": "0.3.1",
    "name": "podman",
    "plugins": [
      {
        "type": "bridge",
        "bridge": "cni0",
        "isGateway": true,
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
            "subnet": "10.88.0.0/16",
            "routes": [
                { "dst": "0.0.0.0/0" }, 
                { "dst": "192.1.1.0/24", "gw": "10.88.0.254" }
            ]
        }
      },
      {
        "type": "route-overwrite",
        "delroutes": [ {
            "dst": "192.1.1.0/24"
        } ],
        "flushgateway": false,
        "flushroutes": false 
      },
      {
        "type": "portmap",
        "capabilities": {
          "portMappings": true
        }
      }
    ]
}
[tohayash@tohayash-lab ~]$ sudo podman run --network=podman -it docker.io/centos/tools:latest bash
[root@725f4724e0a7 /]# ip route
default via 10.88.0.1 dev eth0 
10.88.0.0/16 dev eth0 proto kernel scope link src 10.88.0.39
@williamcaban
Copy link

Since it is not compiling the code or building the containers, the installation can be done directly from github with

oc create -f https://raw.githubusercontent.com/redhat-nfvpe/cni-route-override/master/deployments/daemonset-install.yaml

@elan117
Copy link

elan117 commented Nov 18, 2020

This plug only support CenOS?
Are other system versions supported? For example: Ubuntu .....
Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment