Installation and setting guide of kubernetes using kubeadm

k8s_installation_guide.md

Kubernetes Installation & Setup Guide

https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/

Before Begin

Following document is about installing and setting up a cluster using kubernetes. Basic structure it supports is a cluster with one MASTER and multiple NODES(slave nodes).

• COMMON : You should perform for both MASTER and NODE.
• MASTER : You should perform for MASTER only.
• NODE : You should perform for NODE only.

Prerequisites

Assuming the OS is Ubuntu 16.04 (Xeniel)

• COMMON
  • Install Docker

   $ sudo apt-get update
   $ sudo apt-get install \
     apt-transport-https \
     ca-certificates \
     curl \
     software-properties-common
   $ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
   $ sudo add-apt-repository \
     "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
     $(lsb_release -cs) \
     stable"
   $ sudo apt-get update
   $ sudo apt-get install docker-ce

Install Kubeadm

(https://kubernetes.io/docs/setup/independent/install-kubeadm/)

• COMMON

  • Install Kubeadm, Kubelet, Kubectl

   $ sudo apt-get update && sudo apt-get install -y apt-transport-https
   $ curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
   $ sudo touch /etc/apt/sources.list.d/kubernetes.list 
   $ echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
   $ sudo apt-get update
   $ sudo apt-get install -y kubelet kubeadm kubectl

  • Swapp off

   $ swapoff -a

  • Configure Cgroup Driver Make sure that the following two results match.

   $ docker info | grep -i cgroup

   $ sudo cat /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

  If they don't match, add following line to '/etc/systemd/system/kubelet.service.d/10-kubeadm.conf' Environment="KUBELET_EXTRA_ARGS=--cgroup-driver=cgroupfs"

   $ sudo systemctl daemon-reload
   $ sudo systemctl restart kubelet

• MASTER

  • Initiate Master (Assuming using Calico as a pod network)

   $ sudo kubeadm init --pod-network-cidr=192.168.0.0/16 --ignore-preflight-errors=all
   $ mkdir -p $HOME/.kube
   $ sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
   $ sudo chown $(id -u):$(id -g) $HOME/.kube/config

  Keep final line of result of ‘kubeadm init’. (kubeadm join …). Node can join to cluster using this line

  • Add Pod Network Add-on (Using Calico as a pod network)

   $ kubectl apply -f https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/rbac-kdd.yaml
   $ kubectl apply -f https://docs.projectcalico.org/v3.1/getting-started/kubernetes/installation/hosted/kubernetes-datastore/calico-networking/1.7/calico.yaml

  • Isolate Master

   $ kubectl taint nodes --all node-role.kubernetes.io/master-

• NODE

  • Try to join cluster

   $ sudo kubeadm join <MASTER_IP>:<MASTER_PORT> --token <TOKEN> --discovery-token-ca-cert-hash <HAST> --ignore-preflight-errors=all

Extras

Trouble Shooting

• When initiating/joining a cluster causes '[kubelet-check] It seems like the kubelet isn't running or healthy' error

$ sudo cp /var/lib/kubelet/config.yaml /var/lib/kubelet/config_custom.yaml
$ sudo vi /var/lib/kubelet/config_custom.yaml

Modify failSwapOn to false

$ sudo vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Modify /var/lib/kubelet/config.yaml into /var/lib/kubelet/config_custom.yaml

$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet

Check if kubelet running by sudo systemctl status kubelet

• When 'CrashLoopBackOff (# node(s) had taints that the pod didn't tolerate)' error occurs on CoreDNS pods

$ kubectl edit cm coredns -n kube-system

Replace proxy . /etc/resolv.conf into any other DNS. e.g.)proxy . 8.8.8.8 (Refer to https://coredns.io/plugins/loop/#troubleshooting )

Save and exit

kubectl get pods -n kube-system -oname |grep coredns |xargs kubectl delete -n kube-system

Dashboard Access

Install dashboard as follows

$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml

Run kube proxy

$ kubectl proxy

Access to dashboard with following url

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/.

$ kubectl -n kube-system get service kubernetes-dashboard

$ kubectl -n kube-system get secret
$ kubectl -n kube-system describe secret default-token-*

Copy the token and use it as login token for dashboard

If permission error occurs, refer to This

• TL;DR;

1. Make File 'dashboard-admin.yaml'

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: kubernetes-dashboard
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: kubernetes-dashboard
  namespace: kube-system

2. $ kubectl create -f dashboard-admin.yaml
3. Apply 'dashboard-admin.yaml' file

$ kubectl -n kube-system get secret
$ kubectl -n kube-system describe secret kubernetes-dashboard-token-*

Using Specific NIC for Clustering

• COMMON

$ sudo vi /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Add --node-ip=<Specific IP> And

$ sudo systemctl daemon-reload
$ sudo systemctl restart kubelet

• MASTER

For kubeadm init, add --apiserver-advertise-address=<Specific IP> option