Changing kubernetes kubedns loglevel

kubedns-loglevel.md

Changing System Pods Log Level

Deployment yaml files for kubernetes system services are typically found in /etc/kubernetes/manifests. The yaml files for addons like kubedns, kube proxy, tiller, etc can be found in /etc/kubernetes/addons.

Kube DNS Default

The default kubedns loglevel is 2, are you can see from looking at the logs for a kubedns pod.

$ kubectl logs kube-dns-v20-597689868c-9l29r kubedns  --namespace=kube-system
I0307 20:13:42.757889       1 dns.go:48] version: 1.14.8
I0307 20:13:42.759198       1 server.go:71] Using configuration read from directory: /kube-dns-config with period 10s
I0307 20:13:42.759272       1 server.go:119] FLAG: --alsologtostderr="false"
[...]
I0307 20:13:42.759767       1 server.go:119] FLAG: --logtostderr="true"
I0307 20:13:42.759838       1 server.go:119] FLAG: --stderrthreshold="2"
I0307 20:13:42.759854       1 server.go:119] FLAG: --v="2"

Kubedns configuration

Take a look at /etc/kubernetes/addons/kube-dns-deployment.yaml. You'll find loglevel defined for the 2 pods of the cluster DNS service, kubedns and dnsmasq:

[...]
apiVersion: apps/v1beta1
kind: Deployment
metadata:
  labels:
    kubernetes.io/cluster-service: "true"
    k8s-app: kube-dns
    version: v20
  name: kube-dns-v20
  namespace: kube-system
spec:
[...]
template:
[...]
    spec:
      containers:
      - args:
        - "--domain=cluster.local."
        - "--dns-port=10053"
        - "--v=2"
        - "--config-dir=/kube-dns-config"
        image: k8s-gcrio.azureedge.net/k8s-dns-kube-dns-amd64:1.14.8
        name: kubedns
[...]          
      - args:
        - "-v=2"
        - "-logtostderr"
        image: k8s-gcrio.azureedge.net/k8s-dns-dnsmasq-nanny-amd64:1.14.8
        name: dnsmasq
[...]

Change the Loglevel

By default, both are set to 2. You can change them to 3 or whatever level you desire. Level 3 logs name resolution calls.

Then

$ kubectl apply -f /etc/kubernetes/addons/kube-dns-deployment.yaml
serviceaccount "kube-dns" unchanged
service "kube-dns" unchanged
deployment "kube-dns-v20" configure

Double Check the configuration change

First, check that the kubedns pods restarted:

$ kubectl get pods --namespace=kube-system
NAMESPACE           NAME                                            READY     STATUS    RESTARTS   AGE
kube-system         kube-controller-manager-k8s-master-26765626-0   1/1       Running   0          19h
kube-system         kube-dns-v20-5684bccfdf-7cknr                   3/3       Running   0          1m
kube-system         kube-dns-v20-5684bccfdf-bgd67                   3/3       Running   0          2m
[...]

Then check the logs for the kubedns pod to show your new loglevel

$ kubectl logs  kube-dns-v20-5684bccfdf-7cknr kubedns  --namespace=kube-system
I0308 16:04:07.686059       1 dns.go:48] version: 1.14.8
I0308 16:04:07.687633       1 server.go:71] Using configuration read from directory: /kube-dns-config with period 10s
I0308 16:04:07.688232       1 server.go:119] FLAG: --v="3"

Now you'll see log entries for DNS resolution.