Last active
January 25, 2022 02:54
-
-
Save venkatzgithub/3bc3898ef460a3c3328f92b40e5c6f39 to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Practise json path 3-4 question will be on that. | |
https://medium.com/faun/be-fast-with-kubectl-1-18-ckad-cka-31be00acc443 | |
https://www.youtube.com/watch?v=wgfjXHw7uPs&feature=youtu.be | |
https://docs.linuxfoundation.org/tc-docs/certification/lf-candidate-handbook/exam-user-interface | |
https://training.linuxfoundation.org/cka-program-changes-2020/ | |
https://medium.com/@shekharsarker/changes-with-certified-kubernetes-administrator-cka-certification-2020-9d17a0e15d59 | |
IMP: | |
Deployment - create, verify, scale, expose, rollout, undorollout, history | |
Pod - create, modify, expose, staticPod, Multicontainer pod, troubleshooting | |
initContainer | |
DaemonSet | |
Secret | |
PV and PVC | |
Network Policy | |
Security Context | |
RBAC | |
ETCD Backup | |
Cluster Installation | |
Cluster Troubleshooting | |
jsonpath | |
priority class | |
cheatsheet | |
https://monicabhartiya.com/images/syllabus-comparision-chart.png | |
While going through all the labs, make sure to read the corresponding kubernetes documentation. Since knowing where/when to look in a docs is very important during exam, it will save lot of your time. | |
After you finish entire course with mock test multiple times. Remember to give more stress on these three topics - jsonpath, cluster administration and troubleshooting. | |
a. jsonpath: practice as much queries as you can, there are always few questions in the exam. | |
b. cluster administration: focus on kubeadm cluster installation and upgradation with etcd backup and restore. Remember kubeadm init options - read the entire docs atleast once. | |
c. Cluster troubleshooting: practice troubleshooting labs as much as you can. Know the basics like whats the role of scheduler, controller, etc that will help you while troubleshooting. Capturing the problematic component sooner is the key here to save the time. | |
- First 3 minutes, I configured the vim and added the autocompletion from the K8s documentation. | |
cat << EOV >> ~/.vimrc | |
set ts=2 sts=2 sw=2 expandtab ruler | |
set backspace=indent,eol,start | |
EOV | |
The exam was not very time consuming for me because vim is my native text editor and i am very comfortable with *grep, | |
*awk, sed and troubleshooting in linux in service/user/kernel level is an everyday job to me. | |
sudo -i | |
source <(kubectl completion bash) | |
echo "source <(kubectl completion bash)" >> ~/.bashrc | |
complete -F __start_kubectl k | |
export do="--dry-run=client -o yaml" | |
export now="--force --grace-period=0" | |
alias k="kubectl" | |
alias kcf="k create" | |
alias kdf="k delete $now" | |
alias kdff="kdf -f" | |
alias krp="k run" | |
alias krpy="krp $do" | |
alias krd="k create deployment" | |
alias krdy="krd $do" | |
If you simply want to create a Pod use : | |
krp --image=nging nginx | |
If you want pod with yaml use: | |
krpy --image=nging nginx > nginx-pod.yaml | |
If you simply want to create a Deployment use : | |
krd --image=nging nginx | |
If you want deployment with yaml use: | |
krdy --image=nging nginx > nginx-dep.yaml | |
If you want to delete a Pod use : | |
kdf po nginx | |
If you want to delete a Pod for which yaml is present use : | |
kdff nginx-pod.yaml | |
I wanted to review what should be key approaches for fixing a broken cluster and how components are inter-dependent. I am purposefully leaving some troubleshooting scenarios (pure application related troubleshooting, pure network related troubleshooting) out of the scope of this review. This may be a bit of a dry read, but I just wanted to gather my thoughts together at once and seeking your help in identifying any crucial step/link that I may be missing. | |
1. kubectl is not working (with the default kube-config file in $HOME/.kube/config) | |
a) Verify that the cluster server info looks good (ip and port) | |
kubectl config view | |
Note: Usually kube-api-server default port is 6443. If in doubt, verify with kube-api-server definition file cat /etc/kubernetes/manifests/<<kube-api-server-yaml-file>>. | |
b) Verify that the kube-api-server is working fine. See section "What if kube-api-server is not working" below. | |
2. What if kube-api-server is not working. | |
If kube-api-server is not running fine, most likely kubectl is not working as well. Since, kubectl is not working, we will have to restort to docker commands. | |
i) Verify kube-api-server container is runinng and get the container id | |
docker container ls -a | grep kube-api #shows both live and dead containers | |
Check the logs of kube-api-server for potential issues | |
docker container logs <<container id found from above commands>> | |
If logs indicate that kube-api-server is not able to connect to etcd. Etcd may not be healthy. See section "What if etcd databaes server is not working" section below. | |
Otherwise, issues could be anything from (including, but not limited to) the "Common Error Scenarios" section below. | |
Fix the error and kube-api-server should come up. | |
3. What if etcd database server is not working | |
If the etcd server is not working, most likely its impacting functioning of kube-api-server too and hence kubectl may not be working as well. | |
i) Verify etcd container is runinng and get the container id | |
docker container ls -a | grep etcd #shows both live and dead containers | |
Check the logs of kube-api-server for potential issues | |
docker container logs <<container id found from above commands>> | |
Issues could be anything from (including, but not limited to) the "Common Error Scenarios" section below. | |
Fix the error and etcd-server should come up. | |
4. kube-scheduler is not working | |
i) Check pod events using | |
kubectl -n kube-system describe pod <<kube-scheduler-pod-name>> | |
ii) Check pod logs using | |
kubectl -n kube-system logs <<kube-scheduler-pod-name>> | |
Issues identified from the above, can be anything (including, but not limited to) from the "Common Error Scenarios" section below. | |
5. kube-controller manager is not working | |
i) Check pod events using | |
kubectl -n kube-system describe pod <<kube-controller-pod-name>> | |
ii) Check pod logs using | |
kubectl -n kube-system logs <<kube-controller-pod-name>> | |
Issues identified from the above, can be anything (including, but not limited to) from the "Common Error Scenarios" section below. | |
6. kubelet is not working | |
i) Verify the status of kubelet service | |
systemctl status kubelet.service | |
ii) Check logs for kubelet service | |
journalctl -u kubelet.service | |
iii) If we need to fix something (cert location?) in the kubelet config file, find kubelet config file using either of the following ways - | |
cat /etc/systemd/system/kubelet.*/<<kubeadm-conf>> | |
systemctl status kubelet.service => will display kubelet drop-in files and/or path for kubelet config file, mostly like - /var/lib/kubelet/config.yaml-file | |
iv) If we need to fix something (cluster connection parameters?) in the kubelet kubeconfig file, find kubelet config file using either of the following ways - | |
cat /etc/systemd/system/kubelet.*/<<kubeadm-conf>> | |
systemctl status kubelet.service => will display kubelet drop-in files and/or path for kubelet kubeconfig file, mostly like - /var/lib/kubelet/kubeconfig.yaml-file | |
v) After updating any kubelet config or kubeconfig, | |
systemctl daemon-reload | |
systemctl start/restart kubelet.service | |
vi) Verify kubelet log is reasonably clan - | |
journalctl -u kubelet.service | |
7. Static Pod issues/configurations | |
i) Make sure kubelet is running, because kubelet is responsible for hosting static pods. | |
ii) Make sure the staticPodPath parameters is correctly configured in /var/lib/kubelet/config.yaml file. If not already set, please update the file to have the path. | |
iii) Make sure the staticPodPath (default - /etc/kubernetes/manifests) exists in the relevant node. If doesn't exist, plesae create the dir. | |
8. Common Error Scenarios | |
a) Certificate file name is spelled wrongly in the config command. | |
b) Certificate file can't be found in the configured location because the volume mount has been wrongly directing a different location. | |
c) Correct certificate may not have been used. For example - while calling etcd from api-server, the ca.crt for etcd wasn't used, if there was one. | |
d) Command argument may have executable - cuainsg the pod to restart continiously. | |
e) Pod definition file may have typos in the image name - causing image pull error. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment