Skip to content

Instantly share code, notes, and snippets.

@wingkwong

wingkwong/kubernetes-application-bible-for-developers.md

Last active May 24, 2022 17:18
Show Gist options
  • Save wingkwong/38a2a6d3580a4e7ed8c06c41732dcfa8 to your computer and use it in GitHub Desktop.
Save wingkwong/38a2a6d3580a4e7ed8c06c41732dcfa8 to your computer and use it in GitHub Desktop.
Download ZIP
This is a bible for certified Kubernetes application developers. It covers the features and how to design and build applications to run on Kubernetes.
Raw
kubernetes-application-bible-for-developers.md

Updates on 22 Feb 2020

  • Updated version to 1.17.3-00 for kubelet, kubeadm and kubectl
  • Updated kube-flannel as extensions/v1beta1 was removed

Creating a cluster

# Setup Docker & Kubernetes repositories
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

cat << EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

# Install Docker & Kubernetes
sudo apt-get update
sudo apt-get install -y docker-ce=18.06.1~ce~3-0~ubuntu kubelet=1.17.3-00 kubeadm=1.17.3-00 kubectl=1.17.3-00
sudo apt-mark hold docker-ce kubelet kubeadm kubectl

# Enable iptables bridge call
echo "net.bridge.bridge-nf-call-iptables=1" | sudo tee -a /etc/sysctl.conf

# Make that change take effect immediately
sudo sysctl -p

Setting up the Kube master server

# Initalize the cluster
sudo kubeadm init --pod-network-cidr=10.244.0.0/16

# Setup local kube config
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

# Install Flannel networking
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Setting up the Kube node server

# The command is part of output of kubeadm init running on your master
sudo kubeadm join $controller_private_ip:6443 --token $token --discovery-token-ca-cert-hash $hash

Working with Kubernetes Objects

Kubernetes Objects are persistent entities in the Kubernetes system. Kubernetes uses these entities to represent the state of your cluster.

Ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/kubernetes-objects/

List all the object types

kubectl api-resources -o name

# Some examples
Pod
Node
Service
ServiceAccount
...

Example of a Kubernetes Object : deployment.yml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  selector:
    matchLabels:
      app: nginx
  replicas: 2 # tells deployment to run 2 pods matching the template
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80

Use kubectl apply

kubectl apply -f deployment.yml --record

Working with Pods

A Pod is the basic execution unit of a Kubernetes application–the smallest and simplest unit in the Kubernetes object model that you create or deploy. A Pod represents processes running on your Cluster.

Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/

Example of pod definition: my-pod.yml

apiVersion: v1
kind: Pod
metadata:
  name: myapp-pod
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 3600']
# Create a pod from the definition file in yaml format
kubectl create -f my-pod.yml

# Edit a pod and re-apply a pod definition
kubectl apply -f my-pod.yml

# Another way to edit a pod (Changes will be taken effect immediately)
kubectl edit pod my-pod

# Delete a pod
kubectl delete pod my-pod

Working with Namespace

Kubernetes supports multiple virtual clusters backed by the same physical cluster. These virtual clusters are called namespaces.

Ref： https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/

By default, a Kubernetes cluster will instantiate a default namespace.

NAME      STATUS    AGE
default   Active    13m

Create a namespace

kubectl create ns development

Spcify a namespace in metadata.namespace attribute

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  namespace: my-namespace
  labels:
    app: myapp
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', 'echo Hello World! && sleep 3600']
kubectl create -f my-namespace.yml

Get a list of the namespaces in the cluster

kubectl get namespaces

NAME          STATUS    AGE
development   Active    5m
default       Active    1d
kube-system   Active    1d
kube-public   Active    1d

Specify a namespace when using kubectl get

kubectl get pods -n my-namespace

Specify a namespace when using kubectl describe

kubectl describe pod my-pod -n my-namespace

Working with ConfigMaps

ConfigMaps allow you to decouple configuration artifacts from image content to keep containerized applications portable.

Ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/

Consuming ConfigMap as Environment Variables

apiVersion: v1
kind: ConfigMap
metadata:
   name: my-config-map
data:
   myKey: myValue
   anotherKey: anotherValue
apiVersion: v1
kind: Pod
metadata:
  name: my-configmap-pod
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', "echo $(MY_VAR) && sleep 3600"]
    env:
    - name: MY_VAR
      valueFrom:
        configMapKeyRef:
          name: my-config-map
          key: myKey

Consuming ConfigMap as Mounted Volume

apiVersion: v1
kind: Pod
metadata:
  name: my-configmap-volume-pod
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', "echo $(cat /etc/config/myKey) && sleep 3600"]
    volumeMounts:
      - name: config-volume
        mountPath: /etc/config
  volumes:
    - name: config-volume
      configMap:
        name: my-config-map

Working with SecurityContexts

A security context defines privilege and access control settings for a Pod or Container.

Ref: https://kubernetes.io/docs/tasks/configure-pod-container/security-context/

apiVersion: v1
kind: Pod
metadata:
  name: my-securitycontext-pod
spec:
  securityContext:
    runAsUser: 2001
    fsGroup: 3001
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', "cat /message/message.txt && sleep 3600"]
    volumeMounts:
    - name: message-volume
      mountPath: /message
  volumes:
  - name: message-volume
    hostPath:
      path: /etc/message

Working with Resource Requests

When you specify a Pod, you can optionally specify how much CPU and memory (RAM) each Container needs. When Containers have resource requests specified, the scheduler can make better decisions about which nodes to place Pods on. And when Containers have their limits specified, contention for resources on a node can be handled in a specified manner. For more details about the difference between requests and limits, see Resource QoS.

Ref: https://kubernetes.io/docs/concepts/configuration/manage-compute-resources-container/#resource-requests-and-limits-of-pod-and-container

apiVersion: v1
kind: Pod
metadata:
  name: my-resource-pod
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', 'echo Hello Kubernetes! && sleep 3600']
    resources:
      requests:
        memory: "64Mi"
        cpu: "250m"
      limits:
        memory: "128Mi"
        cpu: "500m"

Working with Secrets

Kubernetes secret objects let you store and manage sensitive information, such as passwords, OAuth tokens, and ssh keys. Putting this information in a secret is safer and more flexible than putting it verbatim in a Pod definition or in a container image.

Ref: https://kubernetes.io/docs/concepts/configuration/secret/

Create a secret using a yaml definition

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
stringData:
  username: YWRtaW4=
  password: MWYyZDFlMmU2N2Rm

For best practice, delete the definition after you create it

kubectl apply -f my-secret.yml
rm my-secret.yml

Pass the sensitive data to containers as an environment variable

apiVersion: v1
kind: Pod
metadata:
  name: my-secret-pod
spec:
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', "echo Hello, Kubernetes! && sleep 3600"]
    env:
      - name: SECRET_USERNAME
        valueFrom:
          secretKeyRef:
            name: mysecret
            key: username
      - name: SECRET_PASSWORD
        valueFrom:
          secretKeyRef:
            name: mysecret
            key: password

Working with ServiceAccounts

A service account provides an identity for processes that run in a Pod.

Ref:

Create a ServiceAccount

kubectl create serviceaccount my-serviceaccount
apiVersion: v1
kind: Pod
metadata:
  name: my-serviceaccount-pod
spec:
  serviceAccountName: my-serviceaccount
  containers:
  - name: myapp-container
    image: busybox
    command: ['sh', '-c', "echo Hello, Kubernetes! && sleep 3600"]

Working with Multi-Container Pods

Multi-container pods provide an opportunity to enhance containers with helper containers that provide additional functionality

Ref:

apiVersion: v1
kind: Pod
metadata:
  name: multi-container-pod
spec:
  containers:
  - name: nginx
    image: nginx:1.15.8
    ports:
    - containerPort: 80
  - name: busybox-sidecar
    image: busybox
    command: ['sh', '-c', 'while true; do sleep 30; done;']

Working with Probes

A Probe is a diagnostic performed periodically by the kubelet on a Container.

livenessProbe: Indicates whether the Container is running. If the liveness probe fails, the kubelet kills the Container, and the Container is subjected to its restart policy. If a Container does not provide a liveness probe, the default state is Success.

apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-exec
spec:
  containers:
  - name: liveness
    image: k8s.gcr.io/busybox
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
    livenessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 5

readinessProbe: Indicates whether the Container is ready to service requests. If the readiness probe fails, the endpoints controller removes the Pod’s IP address from the endpoints of all Services that match the Pod. The default state of readiness before the initial delay is Failure. If a Container does not provide a readiness probe, the default state is Success.

apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-exec
spec:
  containers:
  - name: liveness
    image: k8s.gcr.io/busybox
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
    readinessProbe:
      exec:
        command:
        - cat
        - /tmp/healthy
      initialDelaySeconds: 5
      periodSeconds: 5

startupProbe: Indicates whether the application within the Container is started. All other probes are disabled if a startup probe is provided, until it succeeds. If the startup probe fails, the kubelet kills the Container, and the Container is subjected to its restart policy. If a Container does not provide a startup probe, the default state is Success.

apiVersion: v1
kind: Pod
metadata:
  labels:
    test: liveness
  name: liveness-exec
spec:
  containers:
  - name: liveness
    image: k8s.gcr.io/busybox
    args:
    - /bin/sh
    - -c
    - touch /tmp/healthy; sleep 30; rm -rf /tmp/healthy; sleep 600
    startupProbe:
      httpGet:
        path: /healthz
        port: liveness-port
      failureThreshold: 30
      periodSeconds: 10

Ref:

Working with Container Logging

Ref: https://kubernetes.io/docs/concepts/cluster-administration/logging/

Get the container's logs

kubectl logs <pod name>

Get the specfic container's logs in a multi-container pod

kubectl logs <pod name> -c <container name>

kubectl logs <pod name> > <log name>

Working with Monitoring Applications

Ref: https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/

# Get resource usage for all pods in the default namespace
kubectl top pods
# Get resource usage for a single pod
kubectl top pod <pod name>
# Get resource usage for all pods in a specific namespace
kubectl top pods -n kube-system
# Get resource usage for nodes
kubectl top nodes

Working with Labels, Selectors and Annotations

Ref:

apiVersion: v1
kind: Pod
metadata:
  name: my-production-label-pod
  labels:
    app: my-app
    environment: production
spec:
  containers:
  - name: nginx
    image: nginx
apiVersion: v1
kind: Pod
metadata:
  name: my-development-label-pod
  labels:
    app: my-app
    environment: development
spec:
  containers:
  - name: nginx
    image: nginx
# show labels for each pods
kubectl get pods --show-labels
# get pods using equality-based selectors
kubectl get pods -l environment=production
kubectl get pods -l environment=development
# get pods using inequality-based selectors
kubectl get pods -l environment!=production
# get pods using set-based selectors
kubectl get pods -l 'environment in (development,production)'
# get pods using chained multiple selectors with a comma-delimited list
kubectl get pods -l app=my-app,environment=production
# to view existing labels and annotations
kubectl describe pod <pod name>

Working with Deployment

A Deployment provides declarative updates for Pods and ReplicaSets. You describe a desired state in a Deployment, and the Deployment Controller changes the actual state to the desired state at a controlled rate. You can define Deployments to create new ReplicaSets, or to remove existing Deployments and adopt all their resources with new Deployments.

Ref: https://kubernetes.io/docs/concepts/workloads/controllers/deployment/

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
kubectl get deployments
kubectl get deployment <deployment name>
kubectl describe deployment <deployment name>
kubectl edit deployment <deployment name>
kubectl delete deployment <deployment name>

Working with Rolling Update

apiVersion: apps/v1
kind: Deployment
metadata:
  name: rolling-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.1
        ports:
        - containerPort: 80
# Perform a rolling update
kubectl set image deployment/rolling-deployment nginx=nginx:1.7.9 --record
# Explore the rollout history of the deployment
kubectl rollout history deployment/rolling-deployment
kubectl rollout history deployment/rolling-deployment --revision=2
# Roll back to the previous revision
kubectl rollout undo deployment/rolling-deployment
# Roll back to a specific earlier revision by providing the revision number
kubectl rollout undo deployment/rolling-deployment --to-revision=1

Proportional scaling by setting maxSurge and maxUnavailable in the deployment spec

spec:
  strategy:
    rollingUpdate:
      maxSurge: 3
      maxUnavailable: 2

Working with Jobs and CronJobs

Ref:

This job calculates the first 2000 digits of pi

apiVersion: batch/v1
kind: Job
metadata:
  name: pi
spec:
  template:
    spec:
      containers:
      - name: pi
        image: perl
        command: ["perl",  "-Mbignum=bpi", "-wle", "print bpi(2000)"]
      restartPolicy: Never
  backoffLimit: 4

Check the status of Jobs. It may take several seconds to finish.

kubectl get jobs

By running kubectl get pods, you should see a new pod with STATUS Completed

kubectl get pods

Check the output

kubectl logs <pod_name>

A cronjob example

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: hello
spec:
  schedule: "*/1 * * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox
            args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure

List and check the status of CronJobs

kubectl get cronjobs

Working with Services

Ref:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.7.9
        ports:
        - containerPort: 80
# Get services
kubectl get svc
# Get endpoints
kubectl get endpoints my-service

Working with Network Policies

A network policy is a specification of how groups of pods are allowed to communicate with each other and other network endpoints.

Ref: https://kubernetes.io/docs/concepts/services-networking/network-policies/

In order to use NetworkPolicies in the cluster, we need to have a network plugin called canal

wget -O canal.yaml https://docs.projectcalico.org/v3.5/getting-started/kubernetes/installation/hosted/canal/canal.yaml

kubectl apply -f canal.yaml

Create a sample nginx pod

apiVersion: v1
kind: Pod
metadata:
  name: network-policy-secure-pod
  labels:
    app: secure-app
spec:
  containers:
  - name: nginx
    image: nginx
    ports:
    - containerPort: 80

Create a client pod used to test network access to the nginx pod

apiVersion: v1
kind: Pod
metadata:
  name: network-policy-client-pod
spec:
  containers:
  - name: busybox
    image: radial/busyboxplus:curl
    command: ["/bin/sh", "-c", "while true; do sleep 3600; done"]

Get the cluster ip address of the nginx pod

kubectl get pod network-policy-secure-pod -o wide

By executing the below command, you should see the network is accessible

kubectl exec network-policy-client-pod -- curl <CLUSTER_IP_ADDRESS>

Create a network policy to restricts all access to the secure pod except pods with allow-access: "true" label

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: my-network-policy
spec:
  podSelector:
    matchLabels:
      app: secure-app
  policyTypes:
  - Ingress
  - Egress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          allow-access: "true"
    ports:
    - protocol: TCP
      port: 80
  egress:
  - to:
    - podSelector:
        matchLabels:
          allow-access: "true"
    ports:
    - protocol: TCP
      port: 80

Apply before network policy and execute the curl command again, you should see the network is not accessible

kubectl exec network-policy-client-pod -- curl <CLUSTER_IP_ADDRESS>

Get more info about NetworkPolicy in the cluster

kubectl get networkpolicies
kubectl describe networkpolicy my-network-policy

Edit the client pod

kubectl edit pod network-policy-client-pod

under metadata, add

labels:
  allow-access: "true"

Re-run the curl command again, the secure pod is accessible from the client pod

Working with Volumes

On-disk files in a Container are ephemeral, which presents some problems for non-trivial applications when running in Containers. First, when a Container crashes, kubelet will restart it, but the files will be lost - the Container starts with a clean state. Second, when running Containers together in a Pod it is often necessary to share files between those Containers. The Kubernetes Volume abstraction solves both of these problems.

Ref: https://kubernetes.io/docs/concepts/storage/volumes/

apiVersion: v1
kind: Pod
metadata:
  name: volume-pod
spec:
  containers:
  - image: busybox
    name: busybox
    command: ["/bin/sh", "-c", "while true; do sleep 3600; done"]
    volumeMounts:
    - mountPath: /tmp/storage
      name: my-volume
  volumes:
  - name: my-volume
    emptyDir: {}

Working with PersistentVolumes and PersistentVolumeClaims

PersistentVolumes (PVs) and PersistentVolumeClaims (PVCs) provide a way to easily consume storage resources, especially in the context of a complex production environment that uses multiple storage solutions.

Ref:

kind: PersistentVolume
apiVersion: v1
metadata:
  name: my-pv
spec:
  storageClassName: local-storage
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  hostPath:
    path: "/mnt/data"
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  storageClassName: local-storage
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 512Mi
kubectl get pv
kubectl get pvc
kind: Pod
apiVersion: v1
metadata:
  name: my-pvc-pod
spec:
  containers:
  - name: busybox
    image: busybox
    command: ["/bin/sh", "-c", "while true; do sleep 3600; done"]
    volumeMounts:
    - mountPath: "/mnt/storage"
      name: my-storage
  volumes:
  - name: my-storage
    persistentVolumeClaim:
      claimName: my-pvc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment