Skip to content

Instantly share code, notes, and snippets.

@stolsma

stolsma/1-Install.md

Last active Oct 24, 2018
Embed
What would you like to do?
Centos repo files for installing KVM, docker, kubernetes, Helm and Tiller

Installing KVM, libvirt, Docker-CE and Kubernetes on Centos 7.x

First of all be root while doing this... sudo su

Set hostname

Set the hostname:

hostnamectl set-hostname 'k8s-master'

Installing KVM

Install KVM

yum install qemu-kvm qemu-img virt-manager libvirt libvirt-python libvirt-client virt-install virt-viewer bridge-utils
systemctl start libvirtd
systemctl enable libvirtd
lsmod | grep kvm

If needed, install xwindows for use of graphical virt manager:

sudo yum groupinstall "GNOME Desktop" "Graphical Administration Tools"
sudo ln -sf /lib/systemd/system/runlevel5.target /etc/systemd/system/default.target
reboot`

Before Start creating VMs, let’s first create the bridge interface. Bridge interface is required if you want to access virtual machines from outside of your hypervisor network.

cd /etc/sysconfig/network-scripts/
cp ifcfg-eno1 ifcfg-br0

Edit the Interface file and set followings:

[root@ network-scripts]# vi ifcfg-eno1
TYPE=Ethernet
BOOTPROTO=static
DEVICE=eno1
ONBOOT=yes
BRIDGE=br0

Edit the Bridge file (ifcfg-br0) and set the followings:

[root@ network-scripts]# vi ifcfg-br0
TYPE=Bridge
BOOTPROTO=static
DEVICE=br0
ONBOOT=yes

Replace the IP address and DNS server details as per your setup.

Restart the network Service to enable the bridge interface.

systemctl restart network

Check the Bridge interface using below command:

ip addr show br0

Installing Docker and K8S

Disable SELinux

First we need to disable both SELinux and swap. Issue the following commands:

setenforce 0
sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

Disable swap

Next, disable swap with the following command:

swapoff -a

We must also ensure that swap isn't re-enabled during a reboot on each server. Open up the /etc/fstab and comment out the swap entry like this:

# /dev/mapper/centos-swap swap swap defaults 0 0

vi /etc/fstab

Configure the firewall

At this moment just stop the firewall with:

systemctl stop firewalld

Or load the correct firewall rules for Kubernetes master in the firewall:

firewall-cmd --permanent --add-port=6443/tcp
firewall-cmd --permanent --add-port=2379-2380/tcp
firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10251/tcp
firewall-cmd --permanent --add-port=10252/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --reload

Or load the correct firewall rules for Kubernetes workers in the firewall:

firewall-cmd --permanent --add-port=10250/tcp
firewall-cmd --permanent --add-port=10255/tcp
firewall-cmd --permanent --add-port=30000-32767/tcp
firewall-cmd --permanent --add-port=6783/tcp
firewall-cmd  --reload

Enable br_netfilter

Enable the br_netfilter kernel module. This is done with the following commands:

modprobe br_netfilter
echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

Install Docker-ce

Install the Docker-ce dependencies with the following command:

yum install -y yum-utils device-mapper-persistent-data lvm2

Next, add the Docker-ce repository with the command:

yum-config-manager --add-repo https://gist.githubusercontent.com/stolsma/12457f4db016a86fea631fecee419989/raw/ef412fc4e282a3d83fa216b0be53757ecf1edf37/docker-ce.repo
yum update

Install Docker-ce with the command:

yum install -y docker-ce

Start docker automatically on reboot and also now:

systemctl start docker
systemctl enable docker

Install Kubernetes

First we need to create a repository entry for yum. To do this, issue the following command :

yum-config-manager --add-repo https://gist.githubusercontent.com/stolsma/12457f4db016a86fea631fecee419989/raw/ef412fc4e282a3d83fa216b0be53757ecf1edf37/kubernetes.repo
yum update

Install Kubernetes with the command:

yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes

and start the k8s kubelet deamon with:

systemctl enable kubelet
systemctl start kubelet

Once this part of the installation completes, you could reboot the machine. (TODO: but this is not needed?)

Cgroup changes

Now we need to ensure that both Docker-ce and Kubernetes belong to the same control group (cgroup). By default, Docker should already belong to cgroupfs (you can check this with the command docker info | grep -i cgroup). Add Kubernetes to this too, issue the command:

sed -i 's/cgroup-driver=systemd/cgroup-driver=cgroupfs/g' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

Restart the systemd daemon and the kubelet service with the commands:

systemctl daemon-reload
systemctl restart kubelet

Initialize the Kubernetes cluster

We're now ready to initialize the Kubernetes cluster. This is done on kubemaster (and only on that machine). On kubemaster, issue the command (again, adjusting the IP addresses to fit your needs):

kubeadm init --apiserver-advertise-address=<MASTER_IP> --pod-network-cidr=<POD_NETWORK>/<POD_NETWORK_SUBNET_BITS>

With flannel as CNI plugin:

kubeadm init --apiserver-advertise-address=<MASTER_IP> --pod-network-cidr=10.244.0.0/16

When this completes (it'll take anywhere from 30 seconds to 5 minutes), the output should include the joining command for your nodes.

Once that completes, head over to kube2 and issue the command (adjusting the IP address to fit your needs):

kubeadm join <MASTER_IP>:6443 --token <TOKEN> --discovery-token-ca-cert-hash sha256:<DISCOVERY_TOKEN_HASH>

Where TOKEN and DISCOVERY_TOKEN_HASH are the tokens displayed after the initialization command completes. Those are only 24 hour valid!

If you do not have the token, you can get it by running the following command on the master node:

kubeadm token list

The output is similar to this:

TOKEN                    TTL  EXPIRES              USAGES           DESCRIPTION            EXTRA GROUPS
8ewj1p.9r9hcjoqgajrj4gi  23h  2018-06-12T02:51:28Z authentication,  The default bootstrap  system:
                                                   signing          token generated by     bootstrappers:
                                                                    'kubeadm init'.        kubeadm:
                                                                                           default-node-token

By default, tokens expire after 24 hours. If you are joining a node to the cluster after the current token has expired, you can create a new token by running the following command on the master node:

kubeadm token create

The output is similar to this:

5didvk.d09sbcov8ph2amjw

If you don’t have the value of --discovery-token-ca-cert-hash, you can get it by running the following command chain on the master node:

openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | \
   openssl dgst -sha256 -hex | sed 's/^.* //'

The output is similar to this:

8cb2de97839780a412b93877f8507ad6c94f73add17d5d7058e91741c9d5ec78

Configuring Kubernetes

Before Kubernetes can be used, we must take care of a bit of configuration. Come out of root and issue the following three commands (to create a new .kube configuration directory, copy the necessary configuration file, and give the file the proper ownership):

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Deploy flannel network

First set something in iptables to pass bridged IPv4 traffic to iptables’ chains

sysctl net.bridge.bridge-nf-call-iptables=1

Now we must deploy the flannel network to the cluster with the command:

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Master Isolation

By default, your cluster will not schedule pods on the master for security reasons. If you want to be able to schedule pods on the master, e.g. for a single-machine Kubernetes cluster for development, run:

kubectl taint nodes --all node-role.kubernetes.io/master-

With output looking something like:

node "test-01" untainted
taint "node-role.kubernetes.io/master:" not found
taint "node-role.kubernetes.io/master:" not found

This will remove the node-role.kubernetes.io/master taint from any nodes that have it, including the master node, meaning that the scheduler will then be able to schedule pods everywhere.

Checking your nodes

Once the deploy command completes, you should be able to see both nodes on the master, by issuing the command kubectl get nodes

All ready

Congratulations, you now have a Kubernetes cluster ready for pods.

Remove Kubernetes from your system

sudo kubeadm reset

Install Kubectl and Helm on Ubuntu for Windows subsystem

Install Kubectl

To install kubectl on Ubuntu for Windows Subsystem do:

sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
sudo apt-get update
sudo apt-get install -y kubectl

Controlling your cluster from machines other than the master

In order to get a kubectl on some other computer (e.g. laptop) to talk to your cluster, you need to copy the administrator kubeconfig file from your master to your workstation like this:

# To set config to set of configs:
scp root@<master ip>:/etc/kubernetes/admin.conf .
export  KUBECONFIG=$KUBECONFIG:admin.conf
kubectl get nodes

# to use only a specific conf file do: 
kubectl --kubeconfig ./admin.conf get nodes

Note: The example above assumes SSH access is enabled for root. If that is not the case, you can copy the admin.conf file to be accessible by some other user and scp using that other user instead.

The admin.conf file gives the user superuser privileges over the cluster. This file should be used sparingly. For normal users, it’s recommended to generate an unique credential to which you whitelist privileges. You can do this with the kubeadm alpha phase kubeconfig user --client-name <CN> command. That command will print out a KubeConfig file to STDOUT which you should save to a file and distribute to your user. After that, whitelist privileges by using kubectl create (cluster)rolebinding.

Install Helm and Tiller

Helm is the package manager for Kubernetes. It lets you define, install, and upgrade Kubernetes base applications. For more information about Helm, please the visit official website: https://helm.sh.

To install Helm do:

curl https://raw.githubusercontent.com/helm/helm/master/scripts/get > get_helm.sh
chmod 700 get_helm.sh
./get_helm.sh

After installation run helm init and add RBAC service accounts for Tiller to run as: (see https://medium.com/@amimahloof/how-to-setup-helm-and-tiller-with-rbac-and-namespaces-34bf27f7d3c3 for RBAC)

helm init
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
helm init --service-account tiller --upgrade
[docker-ce-stable]
name=Docker CE Stable - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/stable
enabled=1
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-stable-debuginfo]
name=Docker CE Stable - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/stable
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-stable-source]
name=Docker CE Stable - Sources
baseurl=https://download.docker.com/linux/centos/7/source/stable
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-edge]
name=Docker CE Edge - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/edge
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-edge-debuginfo]
name=Docker CE Edge - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/edge
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-edge-source]
name=Docker CE Edge - Sources
baseurl=https://download.docker.com/linux/centos/7/source/edge
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-test]
name=Docker CE Test - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-test-debuginfo]
name=Docker CE Test - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/test
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-test-source]
name=Docker CE Test - Sources
baseurl=https://download.docker.com/linux/centos/7/source/test
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-nightly]
name=Docker CE Nightly - $basearch
baseurl=https://download.docker.com/linux/centos/7/$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-nightly-debuginfo]
name=Docker CE Nightly - Debuginfo $basearch
baseurl=https://download.docker.com/linux/centos/7/debug-$basearch/nightly
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[docker-ce-nightly-source]
name=Docker CE Nightly - Sources
baseurl=https://download.docker.com/linux/centos/7/source/nightly
enabled=0
gpgcheck=1
gpgkey=https://download.docker.com/linux/centos/gpg
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment