Skip to content

Instantly share code, notes, and snippets.

@islander
Last active September 28, 2020 10:34
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save islander/0a77919da907e671ec968c25f507e51b to your computer and use it in GitHub Desktop.
Save islander/0a77919da907e671ec968c25f507e51b to your computer and use it in GitHub Desktop.
Local Minikube cluster using QEMU hypervisor

Local Minikube cluster using QEMU hypervisor

Script for installing minikube cluster using QEMU/KVM hypervisor.

Supported OS

  • Ubuntu 18.04, 20.04 (tested)
  • Debian 10 (not yet tested)
  • CentOS 7

Prerequisites:

First, install QEMU/Libvirt stack and reboot:

$ sudo apt install qemu-kvm libvirt-bin
$ sudo reboot

Installation

$ ./install-minikube-kvm.sh

This script will install:

References

How to run Minikube on KVM

#!/usr/bin/env bash
###################################################################
# Script Name : install-minikube-kvm.sh
# Description : Install minikube and docker-machine with KVM driver
# Args :
# Author : kiba
# Email : zombie32@gmail.com
###################################################################
DISTR=$(lsb_release -is)
CODENAME=$(lsb_release -cs)
DISTR_VER=$(lsb_release -rs)
DOCKER_MACHINE_LATEST="$(curl -s https://api.github.com/repos/docker/machine/releases/latest | grep "browser_download_url.*Linux-x86_64" | cut -d : -f 2,3 | tr -d \")"
MINIKUBE_LATEST="$(curl -s https://api.github.com/repos/kubernetes/minikube/releases/latest | grep "browser_download_url.*deb" | cut -d : -f 2,3 | tr -d \")"
# format output for shell commands
echocode() {
echo -e "\n\t\$ $@\n"
}
# 0. check dependencies
virsh domcapabilities --virttype kvm >/dev/null 2>&1
if [ $? -ne 0 ]
then
echo "FATAL: please install qemu-kvm and libvirt-bin first."
echocode sudo apt install qemu-kvm libvirt-bin
echo "And reboot."
exit 1
fi
echo -e "## Start installation ##\n"
# 1. install docker-machine
if [ ! -f /usr/local/bin/docker-machine ]
then
curl -L $DOCKER_MACHINE_LATEST >/tmp/docker-machine &&
sudo mv /tmp/docker-machine /usr/local/bin/docker-machine &&
sudo chmod +x /usr/local/bin/docker-machine
fi
# 1a. install docker-machine completions
if [ ! -f /etc/bash_completion.d/docker-machine.bash ]
then
base=https://raw.githubusercontent.com/docker/machine/master
for i in docker-machine-prompt.bash docker-machine-wrapper.bash docker-machine.bash
do
sudo wget "$base/contrib/completion/bash/${i}" -P /etc/bash_completion.d
done
fi
echo "To enable the docker-machine shell prompt, add \$(__docker_machine_ps1) to your PS1 setting in ~/.bashrc."
echo "PS1='[\u@\h \W\$(__docker_machine_ps1)]\\\$ '"
echo ""
# 2 install minikube
if [[ "$(which minikube | wc -l)" == "0" ]]
then
if [[ $DISTR =~ (Debian|Ubuntu) ]]
then
wget $MINIKUBE_LATEST && \
sudo dpkg -i "${MINIKUBE_LATEST##*/}" && \
rm -f "${MINIKUBE_LATEST##*/}"
else
wget https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
chmod +x minikube-linux-amd64
sudo mv minikube-linux-amd64 /usr/local/bin/minikube
fi
fi
# 2a. show ver
minikube version
echo ""
# 3. install kubectl
if [[ "$(which kubectl | wc -l)" == "0" ]]
then
if [[ $DISTR =~ (Debian|Ubuntu) ]]
then
# no bionic repo nowadays
[[ "${DISTR_VER//.}" -ge "1804" ]] && CODENAME="xenial"
sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb https://apt.kubernetes.io/ kubernetes-${CODENAME} main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl
else
curl -LO https://storage.googleapis.com/kubernetes-release/release/`curl -s https://storage.googleapis.com/kubernetes-release/release/stable.txt`/bin/linux/amd64/kubectl
chmod +x kubectl
sudo mv kubectl /usr/local/bin/
fi
fi
# 3a. show ver
kubectl version -o yaml
echo ""
# 3b. install kubectl completion
source <(kubectl completion bash)
if [ ! -s /etc/bash_completion.d/kubectl ]
then
kubectl completion bash | sudo tee -a /etc/bash_completion.d/kubectl >/dev/null
fi
# 4. install docker-machine KVM driver
if [ ! -f /usr/local/bin/docker-machine-driver-kvm2 ]
then
curl -LO https://storage.googleapis.com/minikube/releases/latest/docker-machine-driver-kvm2
chmod +x docker-machine-driver-kvm2
sudo mv docker-machine-driver-kvm2 /usr/local/bin/
fi
# 5. Finish
echo "Done! You can now start your cluster: "
echocode minikube start --vm-driver kvm2
echo "or set kvm driver as default:"
echocode minikube config set vm-driver kvm2

Use your own registry in Minikube

For development needs you may enable local docker registry.

Configuration

Start node and enable registry addon

$ minikube start
$ minikube addons enable registry

You should see registry pods and services:

$ kubectl get pods --all-namespaces | grep regist
kube-system            registry-proxy-46f6z                         1/1     Running   1          7h38m
kube-system            registry-qdlgr                               1/1     Running   1          7h38m

$ kubectl get service -n kube-system | grep regist
kube-system            registry                    ClusterIP   10.96.159.133   <none>        80/TCP                   7h38m

Add registry DNS records to /etc/hosts files:

$ REGISTRY_IP=$(kubectl -n kube-system get svc/registry -o=template={{.spec.clusterIP}})
$ echo $REGISTRY_IP
10.96.159.133
$ minikube ssh -- "echo -e '$REGISTRY_IP\tregistry.kube-system.svc.cluster.local' | sudo tee -a /etc/hosts"
$ minikube ssh -- cat /etc/hosts
127.0.0.1	localhost
127.0.1.1	minikube
10.96.159.133	registry.kube-system.svc.cluster.local
$ kubectl apply -f https://k8s.io/examples/admin/dns/dnsutils.yaml
$ kubectl exec -ti dnsutils -- nslookup registry.kube-system.svc.cluster.local
Server:		10.96.0.10
Address:	10.96.0.10#53

Name:	registry.kube-system.svc.cluster.local
Address: 10.96.159.133

$ echo -e "127.0.0.1\tregistry.kube-system.svc.cluster.local" | sudo tee -a /etc/hosts
10.96.159.133	registry.kube-system.svc.cluster.local

Forward registry to localhost:

$ kubectl port-forward --namespace kube-system service/registry 5000:80 &
$ curl registry.kube-system.svc.cluster.local:5000/v2/_catalog
{"repositories":[]}

Tag and upload image to your registry

$ docker tag devel/myapp:v1 localhost:5000/devel/myapp:v1
$ docker push localhost:5000/devel/myapp:v1
$ curl registry.kube-system.svc.cluster.local:5000/v2/_catalog
{"repositories":["devel/myapp"]}

Using

Write manifest:

$ vim myapp.yaml
apiVersion: v1
kind: Pod
metadata:
  name: myapp
  labels:
    app: myapp
spec:
  restartPolicy: Never
  containers:
  - name: waiter
    image: localhost:5000/devel/myapp:v1
    command: ["/bin/sleep","120"]

Apply it:

$ kubectl apply -f myapp.yaml

Watch pod activity:

$ kubectl get events --field-selector involvedObject.name=myapp
LAST SEEN   TYPE     REASON      OBJECT      MESSAGE
<unknown>   Normal   Scheduled   pod/myapp   Successfully assigned default/myapp to minikube
9m24s       Normal   Pulling     pod/myapp   Pulling image "localhost:5000/devel/myapp:v1"
8m50s       Normal   Pulled      pod/myapp   Successfully pulled image "localhost:5000/devel/myapp:v1"
8m50s       Normal   Created     pod/myapp   Created container waiter
8m50s       Normal   Started     pod/myapp   Started container waiter

$ kubectl get pods
NAME       READY   STATUS    RESTARTS   AGE
myapp      1/1     Running   0          10m

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment