Skip to content

Instantly share code, notes, and snippets.

@venkatzgithub
Last active January 25, 2022 02:54
Show Gist options
  • Save venkatzgithub/3bc3898ef460a3c3328f92b40e5c6f39 to your computer and use it in GitHub Desktop.
Save venkatzgithub/3bc3898ef460a3c3328f92b40e5c6f39 to your computer and use it in GitHub Desktop.
Practise json path 3-4 question will be on that.
https://medium.com/faun/be-fast-with-kubectl-1-18-ckad-cka-31be00acc443
https://www.youtube.com/watch?v=wgfjXHw7uPs&feature=youtu.be
https://docs.linuxfoundation.org/tc-docs/certification/lf-candidate-handbook/exam-user-interface
https://training.linuxfoundation.org/cka-program-changes-2020/
https://medium.com/@shekharsarker/changes-with-certified-kubernetes-administrator-cka-certification-2020-9d17a0e15d59
IMP:
Deployment - create, verify, scale, expose, rollout, undorollout, history
Pod - create, modify, expose, staticPod, Multicontainer pod, troubleshooting
initContainer
DaemonSet
Secret
PV and PVC
Network Policy
Security Context
RBAC
ETCD Backup
Cluster Installation
Cluster Troubleshooting
jsonpath
priority class
cheatsheet
https://monicabhartiya.com/images/syllabus-comparision-chart.png
While going through all the labs, make sure to read the corresponding kubernetes documentation. Since knowing where/when to look in a docs is very important during exam, it will save lot of your time.
After you finish entire course with mock test multiple times. Remember to give more stress on these three topics - jsonpath, cluster administration and troubleshooting.
a. jsonpath: practice as much queries as you can, there are always few questions in the exam.
b. cluster administration: focus on kubeadm cluster installation and upgradation with etcd backup and restore. Remember kubeadm init options - read the entire docs atleast once.
c. Cluster troubleshooting: practice troubleshooting labs as much as you can. Know the basics like whats the role of scheduler, controller, etc that will help you while troubleshooting. Capturing the problematic component sooner is the key here to save the time.
- First 3 minutes, I configured the vim and added the autocompletion from the K8s documentation.
cat << EOV >> ~/.vimrc
set ts=2 sts=2 sw=2 expandtab ruler
set backspace=indent,eol,start
EOV
The exam was not very time consuming for me because vim is my native text editor and i am very comfortable with *grep,
*awk, sed and troubleshooting in linux in service/user/kernel level is an everyday job to me.
sudo -i
source <(kubectl completion bash)
echo "source <(kubectl completion bash)" >> ~/.bashrc
complete -F __start_kubectl k
export do="--dry-run=client -o yaml"
export now="--force --grace-period=0"
alias k="kubectl"
alias kcf="k create"
alias kdf="k delete $now"
alias kdff="kdf -f"
alias krp="k run"
alias krpy="krp $do"
alias krd="k create deployment"
alias krdy="krd $do"
If you simply want to create a Pod use :
krp --image=nging nginx
If you want pod with yaml use:
krpy --image=nging nginx > nginx-pod.yaml
If you simply want to create a Deployment use :
krd --image=nging nginx
If you want deployment with yaml use:
krdy --image=nging nginx > nginx-dep.yaml
If you want to delete a Pod use :
kdf po nginx
If you want to delete a Pod for which yaml is present use :
kdff nginx-pod.yaml
I wanted to review what should be key approaches for fixing a broken cluster and how components are inter-dependent. I am purposefully leaving some troubleshooting scenarios (pure application related troubleshooting, pure network related troubleshooting) out of the scope of this review. This may be a bit of a dry read, but I just wanted to gather my thoughts together at once and seeking your help in identifying any crucial step/link that I may be missing.
1. kubectl is not working (with the default kube-config file in $HOME/.kube/config)
a) Verify that the cluster server info looks good (ip and port)
kubectl config view
Note: Usually kube-api-server default port is 6443. If in doubt, verify with kube-api-server definition file cat /etc/kubernetes/manifests/<<kube-api-server-yaml-file>>.
b) Verify that the kube-api-server is working fine. See section "What if kube-api-server is not working" below.
2. What if kube-api-server is not working.
If kube-api-server is not running fine, most likely kubectl is not working as well. Since, kubectl is not working, we will have to restort to docker commands.
i) Verify kube-api-server container is runinng and get the container id
docker container ls -a | grep kube-api #shows both live and dead containers
Check the logs of kube-api-server for potential issues
docker container logs <<container id found from above commands>>
If logs indicate that kube-api-server is not able to connect to etcd. Etcd may not be healthy. See section "What if etcd databaes server is not working" section below.
Otherwise, issues could be anything from (including, but not limited to) the "Common Error Scenarios" section below.
Fix the error and kube-api-server should come up.
3. What if etcd database server is not working
If the etcd server is not working, most likely its impacting functioning of kube-api-server too and hence kubectl may not be working as well.
i) Verify etcd container is runinng and get the container id
docker container ls -a | grep etcd #shows both live and dead containers
Check the logs of kube-api-server for potential issues
docker container logs <<container id found from above commands>>
Issues could be anything from (including, but not limited to) the "Common Error Scenarios" section below.
Fix the error and etcd-server should come up.
4. kube-scheduler is not working
i) Check pod events using
kubectl -n kube-system describe pod <<kube-scheduler-pod-name>>
ii) Check pod logs using
kubectl -n kube-system logs <<kube-scheduler-pod-name>>
Issues identified from the above, can be anything (including, but not limited to) from the "Common Error Scenarios" section below.
5. kube-controller manager is not working
i) Check pod events using
kubectl -n kube-system describe pod <<kube-controller-pod-name>>
ii) Check pod logs using
kubectl -n kube-system logs <<kube-controller-pod-name>>
Issues identified from the above, can be anything (including, but not limited to) from the "Common Error Scenarios" section below.
6. kubelet is not working
i) Verify the status of kubelet service
systemctl status kubelet.service
ii) Check logs for kubelet service
journalctl -u kubelet.service
iii) If we need to fix something (cert location?) in the kubelet config file, find kubelet config file using either of the following ways -
cat /etc/systemd/system/kubelet.*/<<kubeadm-conf>>
systemctl status kubelet.service => will display kubelet drop-in files and/or path for kubelet config file, mostly like - /var/lib/kubelet/config.yaml-file
iv) If we need to fix something (cluster connection parameters?) in the kubelet kubeconfig file, find kubelet config file using either of the following ways -
cat /etc/systemd/system/kubelet.*/<<kubeadm-conf>>
systemctl status kubelet.service => will display kubelet drop-in files and/or path for kubelet kubeconfig file, mostly like - /var/lib/kubelet/kubeconfig.yaml-file
v) After updating any kubelet config or kubeconfig,
systemctl daemon-reload
systemctl start/restart kubelet.service
vi) Verify kubelet log is reasonably clan -
journalctl -u kubelet.service
7. Static Pod issues/configurations
i) Make sure kubelet is running, because kubelet is responsible for hosting static pods.
ii) Make sure the staticPodPath parameters is correctly configured in /var/lib/kubelet/config.yaml file. If not already set, please update the file to have the path.
iii) Make sure the staticPodPath (default - /etc/kubernetes/manifests) exists in the relevant node. If doesn't exist, plesae create the dir.
8. Common Error Scenarios
a) Certificate file name is spelled wrongly in the config command.
b) Certificate file can't be found in the configured location because the volume mount has been wrongly directing a different location.
c) Correct certificate may not have been used. For example - while calling etcd from api-server, the ca.crt for etcd wasn't used, if there was one.
d) Command argument may have executable - cuainsg the pod to restart continiously.
e) Pod definition file may have typos in the image name - causing image pull error.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment