dougbtv

SR-IOV with OpenShift Virtualization

We're going to use SR-IOV technology with OpenShift Virtualization. Today we're going to cover looking at the SR-IOV network attachment definitions, spinning up VMs associated with those, and then we'll make a SIP phone call over the VMs (because we're all bored of a ping test!)

Notably, this gist does not cover installing the SR-IOV Operator, configuring the SR-IOV operator.

The NetworkAttachmentDefinition (which defines an intent to attach additional networks to pods in Kubernetes, which you typically see by the presence of an additional network interface) that we use here is the one [provided by default by the SR-IOV network operator](https://docs.openshift.com/container-platform/4.5/netw

dougbtv

Whereabouts also accepts all options from the static IPAM plugin:

https://github.com/containernetworking/plugins/tree/master/plugins/ipam/static

dougbtv

Istio + Multus CNI: Annotation clobbering, replication and fix

This details a reference deployment of Istio w/ Multus CNI to demonstrate a problem where annotations are being clobbered by the Istio webhook. It also provides a patch and workflow for a possible fix.

This article first demonstrates how to reproduce the article, then proposes a patch, and demonstrates a way to build and deploy Istio with the modified code.

NOTE: Ignore the 1.5.1 through the install, I replicate it with latest (Nov 2021), and provide further steps following the rest of the installation.

Suggested system

dougbtv

This is a demonstration of the Bridge CNI plugin, plus the static IPAM CNI plugin.

Requirements

• A recently installed OpenShift (4.2/4.3) installed cluster.

Demo!

Firstly, we'll label a node so that we have two pods going to the same node.

dougbtv

Interactively running the cluster-network-operator

This describes how to run the cluster-network-operator for development purposes, using the ./hack/run-locally.sh script.

You'll need:

• A recent install of openshift, here's Doug's cheat sheet
• A clone of the cluster-network-operator.

Once that's in place you'll need to know:

dougbtv

Installation and usage of cni-route-override

Covers installing & configuring an example cni-route-override instance.

This example simply adds a static route.

Install route-override binaries

Install using the provided daemonset, this will copy the route-override binary to the /opt/cni/bin on each machine in your Kubernetes cluster.

dougbtv

Using Whereabouts + Multus to add an additional route on an additional network interface...

Install an etcd for data storage for Whereabouts.

$ git clone https://github.com/coreos/etcd-operator.git
$ cd etcd-operator/
$ example/rbac/create_role.sh
$ kubectl create -f example/deployment.yaml
$ watch -n1 kubectl get pods -o wide --all-namespaces

dougbtv

#!/bin/bash

get_sriov_pci_root_addresses() {
  for dir in $(find /sys/devices/ -name sriov_totalvfs -exec dirname {} \;); do
    if [ $(cat $dir/sriov_numvfs) -gt 0 ]; then
      # use perl because sed doesn't support non-greedy matching
      basename $dir | perl -pe 's|(.*?:)(.*)|\2|'
    fi
  done
}

dougbtv

A demo of the Whereabouts plugin.

Pre-reqs.

• Install Multus CNI.
• Install etcd operator & etcd.

Demo

Clone Whereabouts and install it...

dougbtv

stop the cluster version operator

[core@ip-10-0-7-170 ~]$ oc get deployments --namespace=openshift-cluster-version
[core@ip-10-0-7-170 ~]$ oc scale deployment cluster-version-operator --replicas=0 --namespace=openshift-cluster-version

Edit our deployment

Check out the current deployment...