Setup Kubernetes Using Kubeadm Cheatsheet (It Just Work!)

README.md

Kubernetes Production Cluster Cheatsheet

Bootstraping clusters with kubeadm

Precheck

https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/install-kubeadm/#before-you-begin

• One or more machines running one of:
  • Ubuntu 16.04+
  • Debian 9+
  • CentOS 7
  • Red Hat Enterprise Linux (RHEL) 7
  • Fedora 25+
  • HypriotOS v1.0.1+
  • Container Linux (tested with 1800.6.0)
• 2 GB or more of RAM per machine (any less will leave little room for your apps)
• 2 CPUs or more
• Full network connectivity between all machines in the cluster (public or private network is fine)
• Unique hostname, MAC address, and product_uuid for every node. See here for more details.
• Certain ports are open on your machines. See here for more details.
• Swap disabled. You MUST disable swap in order for the kubelet to work properly.

Docker as Container Runtime

https://docs.docker.com/engine/install/ubuntu/#install-using-the-repository

Allow apt to use repository over HTTPS

sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

Add Docker’s official GPG key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add repository

sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

Install

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

Verify

sudo docker run hello-world

Kubeadm

Add GPG key

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

Add repository

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

Install

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl

Hold update

Warning: These instructions exclude all Kubernetes packages from any system upgrades. This is because kubeadm and Kubernetes require special attention to upgrade.

sudo apt-mark hold kubelet kubeadm kubectl

Verify

All version must match. From the docs:

kubeadm will not install or manage kubelet or kubectl for you, so you will need to ensure they match the version of the Kubernetes control plane you want kubeadm to install for you.

kubeadm version
kubelet --version
kubectl version

server.md

Server 1

$ cat /etc/*release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ lscpu 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       46 bits physical, 48 bits virtual
CPU(s):              2
On-line CPU(s) list: 0,1
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           2
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               13
Model name:          QEMU Virtual CPU version 2.5+
Stepping:            3
CPU MHz:             2399.998
BogoMIPS:            4799.99
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            4096K
L3 cache:            16384K
NUMA node0 CPU(s):   0,1
Flags:               fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_l$ pti

$ cat /proc/meminfo 
MemTotal:        8170208 kB
MemFree:         5447764 kB
MemAvailable:    7607444 kB
...

Server 2

$ cat /etc/*release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ lscpu 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       46 bits physical, 48 bits virtual
CPU(s):              3
On-line CPU(s) list: 0-2
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           3
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               13
Model name:          QEMU Virtual CPU version 2.5+
Stepping:            3
CPU MHz:             2400.004
BogoMIPS:            4800.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            4096K
L3 cache:            16384K
NUMA node0 CPU(s):   0-2
Flags:               fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm pti

$ cat /proc/meminfo 
MemTotal:       12298788 kB
MemFree:         9592772 kB
MemAvailable:   11662644 kB
...

Server 3

$ cat /etc/*release
PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

$ lscpu 
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
Address sizes:       46 bits physical, 48 bits virtual
CPU(s):              3
On-line CPU(s) list: 0-2
Thread(s) per core:  1
Core(s) per socket:  1
Socket(s):           3
NUMA node(s):        1
Vendor ID:           GenuineIntel
CPU family:          6
Model:               13
Model name:          QEMU Virtual CPU version 2.5+
Stepping:            3
CPU MHz:             2300.000
BogoMIPS:            4600.00
Hypervisor vendor:   KVM
Virtualization type: full
L1d cache:           32K
L1i cache:           32K
L2 cache:            4096K
L3 cache:            16384K
NUMA node0 CPU(s):   0-2
Flags:               fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 syscall nx lm rep_good nopl xtopology cpuid tsc_known_freq pni cx16 x2apic hypervisor lahf_lm pti

$ cat /proc/meminfo 
MemTotal:       12298756 kB
MemFree:         9494348 kB
MemAvailable:   11650000 kB
...