Skip to content

Instantly share code, notes, and snippets.

@spiarh

spiarh/override-kube.md

Last active November 5, 2019 14:29
Show Gist options
  • Save spiarh/8c52e15f9e89313620591f4fe7d62adb to your computer and use it in GitHub Desktop.
Save spiarh/8c52e15f9e89313620591f4fe7d62adb to your computer and use it in GitHub Desktop.
Download ZIP
Raw
override-kube.md

These commands must be run on all the masters

  • override kube-controller-manager ExecStart in systemd service
# mkdir /etc/systemd/system/kube-controller-manager.service.d
# cat > /etc/systemd/system/kube-controller-manager.service.d/override.conf<<EOF
[Service]
ExecStart=
ExecStart=/usr/bin/hyperkube controller-manager \\
            \$KUBE_LOGTOSTDERR \\
            \$KUBE_LOG_LEVEL \\
            \$KUBE_MASTER \\
            \$KUBE_FEATURE_GATES \\
            \$KUBE_CONTROLLER_MANAGER_ARGS \\
            --horizontal-pod-autoscaler-use-rest-clients=false
EOF
# cat /etc/systemd/system/kube-controller-manager.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/hyperkube controller-manager \
            $KUBE_LOGTOSTDERR \
            $KUBE_LOG_LEVEL \
            $KUBE_MASTER \
            $KUBE_FEATURE_GATES \
            $KUBE_CONTROLLER_MANAGER_ARGS \
            --horizontal-pod-autoscaler-use-rest-clients=false
  • override kube-apiserver ExecStart in systemd service
# mkdir /etc/systemd/system/kube-apiserver.service.d
# cat > /etc/systemd/system/kube-apiserver.service.d/override.conf<<EOF
[Service]
ExecStart=
ExecStart=/usr/bin/hyperkube apiserver \\
            \$KUBE_LOGTOSTDERR \\
            \$KUBE_LOG_LEVEL \\
            \$KUBE_ETCD_SERVERS \\
            \$KUBE_API_ADDRESS \\
            \$KUBE_API_PORT \\
            \$KUBELET_PORT \\
            \$KUBE_ALLOW_PRIV \\
            \$KUBE_SERVICE_ADDRESSES \\
            \$KUBE_FEATURE_GATES \\
            \$KUBE_API_ARGS \\
            --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds,PodSecurityPolicy,MutatingAdmissionWebhook,ValidatingAdmissionWebhook
EOF
# cat /etc/systemd/system/kube-apiserver.service.d/override.conf
[Service]
ExecStart=
ExecStart=/usr/bin/hyperkube apiserver \
            $KUBE_LOGTOSTDERR \
            $KUBE_LOG_LEVEL \
            $KUBE_ETCD_SERVERS \
            $KUBE_API_ADDRESS \
            $KUBE_API_PORT \
            $KUBELET_PORT \
            $KUBE_ALLOW_PRIV \
            $KUBE_SERVICE_ADDRESSES \
            $KUBE_FEATURE_GATES \
            $KUBE_API_ARGS \
            --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds,PodSecurityPolicy,MutatingAdmissionWebhook,ValidatingAdmissionWebhook
  • Reload systemd daemon
# systemctl daemon-reload
  • Check that the service are correctly overriden (extended)
# systemd-delta --type=extended|grep kube
[EXTENDED]   /usr/lib/systemd/system/kube-apiserver.service → /etc/systemd/system/kube-apiserver.service.d/override.conf
[EXTENDED]   /usr/lib/systemd/system/kube-controller-manager.service → /etc/systemd/system/kube-controller-manager.service.d/override.conf
  • Restart services
# systemctl restart kube-apiserver
# systemctl restart kube-controller-manager
  • Check the values are taken into account
# ps aux| grep controller-manager| grep --color autoscaler
kube     31627  2.8  3.4 649344 140396 ?       Ssl  14:41   0:05 /usr/bin/hyperkube controller-manager --logtostderr=true --v=2 --kubeconfig=/var/lib/kubelet/kube-controller-mgr-config --use-service-account-credentials --leader-elect=true --cluster-name=kubernetes --cluster-cidr=172.16.0.0/13 --service-account-private-key-file=/etc/pki/sa.key --service-cluster-ip-range=172.24.0.0/16 --allocate-node-cidrs=true --node-cidr-mask-size=23 --root-ca-file=/etc/pki/trust/anchors/SUSE_CaaSP_CA.crt --horizontal-pod-autoscaler-use-rest-clients=false
# ps aux| grep apiserver| grep --color 'MutatingAdmissionWebhook,ValidatingAdmissionWebhook'
kube     31571 11.2  9.0 684616 363544 ?       Ssl  14:41   0:29 /usr/bin/hyperkube apiserver --logtostderr=true --v=2 --etcd-cafile=/etc/pki/trust/anchors/SUSE_CaaSP_CA.crt --etcd-certfile=/etc/pki/kube-apiserver.crt --etcd-keyfile=/etc/pki/kube-apiserver.key --etcd-servers=https://vm152211:2379,https://vm152234:2379,https://vm153148:2379 --insecure-bind-address=127.0.0.1 --bind-address=0.0.0.0 --insecure-port=8080 --secure-port=6444 --allow-privileged=true --service-cluster-ip-range=172.24.0.0/16 --advertise-address=10.84.152.211 --apiserver-count=1 --tls-cert-file=/etc/pki/kube-apiserver.crt --tls-private-key-file=/etc/pki/kube-apiserver.key --tls-ca-file=/etc/pki/trust/anchors/SUSE_CaaSP_CA.crt --cert-dir=/etc/pki --requestheader-username-headers=X-Remote-User --requestheader-group-headers=X-Remote-Group --requestheader-extra-headers-prefix=X-Remote-Extra --requestheader-client-ca-file=/etc/pki/trust/anchors/SUSE_CaaSP_CA.crt --storage-backend=etcd2 --storage-media-type=application/json --service-account-key-file=/etc/pki/sa.key --service-account-lookup=true --runtime-config=admissionregistration.k8s.io/v1alpha1 --authorization-mode=Node,RBAC --oidc-issuer-url=https://vm152211.qa.prv.suse.net:32000 --oidc-client-id=kubernetes --oidc-ca-file=/etc/pki/trust/anchors/SUSE_CaaSP_CA.crt --oidc-username-claim=email --oidc-groups-claim=groups --admission-control=Initializers,NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,PersistentVolumeLabel,DefaultStorageClass,ResourceQuota,DefaultTolerationSeconds,PodSecurityPolicy,MutatingAdmissionWebhook,ValidatingAdmissionWebhook
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment