rke2-multus.md

Multus

update rke2 config

aka install
add the following to the config.yaml from https://docs.rke2.io/install/network_options#using-multus

# /etc/rancher/rke2/config.yaml
cni:
- multus
- canal

to air gap pull rancher/hardened-multus-cni:v4.0.2-build20230811

valdiate install

validate with kubectl get pods -A | grep -i multus-ds

create macvlan config

From https://github.com/k8snetworkplumbingwg/multus-cni/blob/master/docs/quickstart.md#storing-a-configuration-as-a-custom-resource

create NetworkAttachmentDefinition for local network.

cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf
spec:
  config: '{
      "cniVersion": "0.3.1",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.1.0/24",
        "rangeStart": "192.168.1.200",
        "rangeEnd": "192.168.1.216"
      }
    }'
EOF

run test pod

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: samplepod
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
  containers:
  - name: samplepod
    command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: alpine
EOF

get network config from test pod

kubectl exec -it samplepod -- ip a

Moar Fun

Good article : https://devopstales.github.io/kubernetes/multus/

for fun

DHCP anyone? Keep in mind that nohup /opt/cni/bin/dhcp daemon & needs to be running on the control node for DHCP to be passing into the pod.

cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-dhcp
spec:
  config: '{
      "cniVersion": "0.3.1",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": { "type": "dhcp" }
    }'
EOF

and

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: dhcp
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-dhcp
spec:
  containers:
  - name: dhcp
    command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: alpine
EOF

get ip kubectl exec -it dhcp -- ip a and now ping it from an external device.

Or nginx

cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-dhcp
spec:
  containers:
  - name: nginx
    image: nginx
EOF

And we can check for the 192.168.1.0/24 address with kubectl describe pod nginx

z_ipvlan_example.md

ipvlan on ubuntu with single nic

cat <<EOF | kubectl create -f -
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: ipvlan-def
spec:
  config: '{
      "cniVersion": "0.3.1",
      "type": "ipvlan",
      "master": "enp1s0",
      "mode": "l2",
      "ipam": { "type": "static" }
    }'
EOF


cat <<EOF | kubectl create -f -
apiVersion: v1
kind: Pod
metadata:
  name: nginx
  annotations:
    k8s.v1.cni.cncf.io/networks: '[{ "name": "ipvlan-def", "ips": [ "192.168.1.202/24" ] }]'
spec:
  containers:
  - name: nginx
    image: nginx
EOF

for @technotim

my 2 cents. Start with Rocky, remove firewalld, and stick with the default canal. Change only when it truly makes sense.