dougbtv/README.md

## README.md

      
    Raw
  

              README.md
            
          
    Using Whereabouts + Multus to add an additional route on an additional network interface...

Install an etcd for data storage for Whereabouts.
$ git clone https://github.com/coreos/etcd-operator.git
$ cd etcd-operator/
$ example/rbac/create_role.sh
$ kubectl create -f example/deployment.yaml
$ watch -n1 kubectl get pods -o wide  --all-namespaces
$ # wait until the etcd deployment is fully up. Else, next step will fail...
$ kubectl create -f example/example-etcd-cluster.yaml 

Get the IP address for the etcd service...
$ kubectl get svc | grep "etcd-cluster-client"
example-etcd-cluster-client   ClusterIP   10.107.23.76   <none>        2379/TCP            35m

Note, in this case it's 10.107.23.76.
Install Whereabouts...
$ git clone https://github.com/dougbtv/whereabouts && cd whereabouts
$ kubectl apply -f ./doc/daemonset-install.yaml
$ watch -n1 kubectl get pods -o wide  --all-namespaces

Wait for the whereabouts pods to come up. Optionally, verify that there's a Whereabouts binary on disk...
$ ls /opt/cni/bin/whereabouts 
/opt/cni/bin/whereabouts

Create a custom resource based on Whereabouts. In this example I use macvlan, but, you could use anything that will also use an IPAM plugin...
NOTE: You must change the etcd_host to the IP you got above. You'll also likely tailor the range as well as the routes. You may also omit the log_file and log_level, too.
cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: whereabouts-route
spec:
  config: '{
      "cniVersion": "0.3.0",
      "name": "whereaboutsexample",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "whereabouts",
        "etcd_host": "10.107.23.76:2379",
        "range": "192.168.3.190/28",
        "log_file" : "/tmp/whereabouts.log",
        "log_level" : "debug",
        "routes": [
          { "dst": "192.168.3.0/24" },
          { "dst": "192.168.4.0/24" }
        ]
      }
}'
EOF

NOTE: Whereabouts also allows all of the static CNI options which are documented in the static CNI plugin README.
Now, create a pod that references that custom resource via annotation, such as...
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: whereabouts-route-sample
  annotations:
    k8s.v1.cni.cncf.io/networks: whereabouts-route
spec:
  containers:
  - name: whereabouts-route-sample
    command: ["/bin/bash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: dougbtv/centos-network
    securityContext:
      privileged: true
EOF

Now you can see that there has been an additional route added...
[centos@kube-netmachine-master ~]$ kubectl exec -it whereabouts-route-sample -- /bin/bash
[root@whereabouts-route-sample /]# ip route
default via 10.244.1.1 dev eth0 
10.244.0.0/16 via 10.244.1.1 dev eth0 
10.244.1.0/24 dev eth0  proto kernel  scope link  src 10.244.1.162 
192.168.3.0/24 dev net1 
192.168.3.176/28 dev net1  proto kernel  scope link  src 192.168.3.176 
192.168.4.0/24 dev net1 

[root@whereabouts-route-sample /]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
3: eth0@if163: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP 
    link/ether ee:b3:53:43:3d:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 10.244.1.162/24 scope global eth0
       valid_lft forever preferred_lft forever
4: net1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN 
    link/ether 06:82:fb:b3:e2:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.3.176/28 scope global net1
       valid_lft forever preferred_lft forever