Install an etcd for data storage for Whereabouts.
$ git clone https://github.com/coreos/etcd-operator.git
$ cd etcd-operator/
$ example/rbac/create_role.sh
$ kubectl create -f example/deployment.yaml
$ watch -n1 kubectl get pods -o wide --all-namespaces
$ # wait until the etcd deployment is fully up. Else, next step will fail...
$ kubectl create -f example/example-etcd-cluster.yaml
Get the IP address for the etcd service...
$ kubectl get svc | grep "etcd-cluster-client"
example-etcd-cluster-client ClusterIP 10.107.23.76 <none> 2379/TCP 35m
Note, in this case it's 10.107.23.76
.
Install Whereabouts...
$ git clone https://github.com/dougbtv/whereabouts && cd whereabouts
$ kubectl apply -f ./doc/daemonset-install.yaml
$ watch -n1 kubectl get pods -o wide --all-namespaces
Wait for the whereabouts pods to come up. Optionally, verify that there's a Whereabouts binary on disk...
$ ls /opt/cni/bin/whereabouts
/opt/cni/bin/whereabouts
Create a custom resource based on Whereabouts. In this example I use macvlan, but, you could use anything that will also use an IPAM plugin...
NOTE: You must change the etcd_host
to the IP you got above. You'll also likely tailor the range
as well as the routes
. You may also omit the log_file
and log_level
, too.
cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: whereabouts-route
spec:
config: '{
"cniVersion": "0.3.0",
"name": "whereaboutsexample",
"type": "macvlan",
"master": "eth0",
"mode": "bridge",
"ipam": {
"type": "whereabouts",
"etcd_host": "10.107.23.76:2379",
"range": "192.168.3.190/28",
"log_file" : "/tmp/whereabouts.log",
"log_level" : "debug",
"routes": [
{ "dst": "192.168.3.0/24" },
{ "dst": "192.168.4.0/24" }
]
}
}'
EOF
NOTE: Whereabouts also allows all of the static CNI options which are documented in the static CNI plugin README.
Now, create a pod that references that custom resource via annotation, such as...
cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
name: whereabouts-route-sample
annotations:
k8s.v1.cni.cncf.io/networks: whereabouts-route
spec:
containers:
- name: whereabouts-route-sample
command: ["/bin/bash", "-c", "trap : TERM INT; sleep infinity & wait"]
image: dougbtv/centos-network
securityContext:
privileged: true
EOF
Now you can see that there has been an additional route added...
[centos@kube-netmachine-master ~]$ kubectl exec -it whereabouts-route-sample -- /bin/bash
[root@whereabouts-route-sample /]# ip route
default via 10.244.1.1 dev eth0
10.244.0.0/16 via 10.244.1.1 dev eth0
10.244.1.0/24 dev eth0 proto kernel scope link src 10.244.1.162
192.168.3.0/24 dev net1
192.168.3.176/28 dev net1 proto kernel scope link src 192.168.3.176
192.168.4.0/24 dev net1
[root@whereabouts-route-sample /]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
3: eth0@if163: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP
link/ether ee:b3:53:43:3d:5b brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 10.244.1.162/24 scope global eth0
valid_lft forever preferred_lft forever
4: net1@if2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN
link/ether 06:82:fb:b3:e2:3e brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet 192.168.3.176/28 scope global net1
valid_lft forever preferred_lft forever
unable to add default route for multiple interface.
output: