Skip to content

Instantly share code, notes, and snippets.

@thospfuller
Last active November 30, 2020 02:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save thospfuller/d0d918e0b9fdb719a34d3d355b0886bb to your computer and use it in GitHub Desktop.
Save thospfuller/d0d918e0b9fdb719a34d3d355b0886bb to your computer and use it in GitHub Desktop.
An example components.yaml required for installing the Kubernetes Metrics Server on Minikube running in Ubuntu.
#
# Jump to line ~ 141 to ~ 215 for the command that pertains to this change.
#
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: metrics-server
name: metrics-server
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
k8s-app: metrics-server
rbac.authorization.k8s.io/aggregate-to-admin: "true"
rbac.authorization.k8s.io/aggregate-to-edit: "true"
rbac.authorization.k8s.io/aggregate-to-view: "true"
name: system:aggregated-metrics-reader
rules:
- apiGroups:
- metrics.k8s.io
resources:
- pods
- nodes
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
labels:
k8s-app: metrics-server
name: system:metrics-server
rules:
- apiGroups:
- ""
resources:
- pods
- nodes
- nodes/stats
- namespaces
- configmaps
verbs:
- get
- list
- watch
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
labels:
k8s-app: metrics-server
name: metrics-server-auth-reader
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: extension-apiserver-authentication-reader
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
k8s-app: metrics-server
name: metrics-server:system:auth-delegator
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
labels:
k8s-app: metrics-server
name: system:metrics-server
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:metrics-server
subjects:
- kind: ServiceAccount
name: metrics-server
namespace: kube-system
---
apiVersion: v1
kind: Service
metadata:
labels:
k8s-app: metrics-server
name: metrics-server
namespace: kube-system
spec:
ports:
- name: https
port: 443
protocol: TCP
targetPort: https
selector:
k8s-app: metrics-server
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
k8s-app: metrics-server
name: metrics-server
namespace: kube-system
spec:
selector:
matchLabels:
k8s-app: metrics-server
strategy:
rollingUpdate:
maxUnavailable: 0
template:
metadata:
labels:
k8s-app: metrics-server
spec:
containers:
- args:
- --cert-dir=/tmp
- --secure-port=4443
- --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname
- --kubelet-use-node-status-port
image: k8s.gcr.io/metrics-server/metrics-server:v0.4.1
imagePullPolicy: IfNotPresent
#
# Precondition: Metrics Server installation attempted on Minikube (on Ubuntu) as per:
#
# https://github.com/kubernetes-sigs/metrics-server
#
# Specifically, the instructions suggest running the following in order to install the Metrics Server:
#
# kubectl apply -f https://github.com/kubernetes-sigs/metrics-server/releases/latest/download/components.yaml
#
# and running:
#
# kubectl top nodes
#
# results in:
#
# "Error from server (ServiceUnavailable): the server is currently unable to handle the request (get nodes.metrics.k8s.io)"
#
# and running:
#
# kubectl get deployment metrics-server -n kube-system
#
# results in something that looks like:
#
# NAME READY UP-TO-DATE AVAILABLE AGE
# metrics-server 1/1 1 0 6m
#
# -----
#
# We can see in the log that there's a problem:
#
# kubectl logs -n kube-system deploy/metrics-server
#
# E1129 22:08:56.736988 1 server.go:132] unable to fully scrape metrics: [unable to fully scrape metrics from node minikube: unable to fetch metrics from node minikube: Get "https://192.168.99.100:10250/stats/summary?only_cpu_and_memory=true": x509: cannot validate certificate for 192.168.99.100 because it doesn't contain any IP SANs, unable to fully scrape metrics from node m01: unable to fetch metrics from node m01: Get "https://192.168.99.100:10250/stats/summary?only_cpu_and_memory=true": x509: cannot validate certificate for 192.168.99.100 because it doesn't contain any IP SANs]
# I1129 22:08:56.748352 1 requestheader_controller.go:169] Starting RequestHeaderAuthRequestController
# I1129 22:08:56.748378 1 shared_informer.go:240] Waiting for caches to sync for RequestHeaderAuthRequestController
# I1129 22:08:56.748400 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::client-ca-file
# I1129 22:08:56.748404 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
# I1129 22:08:56.748414 1 configmap_cafile_content.go:202] Starting client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
# I1129 22:08:56.748417 1 shared_informer.go:240] Waiting for caches to sync for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
# I1129 22:08:56.749024 1 secure_serving.go:197] Serving securely on [::]:4443
# I1129 22:08:56.749096 1 dynamic_serving_content.go:130] Starting serving-cert::/tmp/apiserver.crt::/tmp/apiserver.key
# I1129 22:08:56.749163 1 tlsconfig.go:240] Starting DynamicServingCertificateController
# I1129 22:08:56.848729 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file
# I1129 22:08:56.848758 1 shared_informer.go:247] Caches are synced for client-ca::kube-system::extension-apiserver-authentication::client-ca-file
# I1129 22:08:56.849090 1 shared_informer.go:247] Caches are synced for RequestHeaderAuthRequestController
#
# -----
#
# The solution at the link below will remedy this issue:
#
# https://github.com/kubernetes-sigs/metrics-server/issues/196#issuecomment-451061841
#
# And we'll need to either modify the component.yaml file with the command below and
# apply the file or we can edit the current configuration using:
#
# kubectl edit deployment -n kube-system metrics-server
#
# Then execute:
#
# minikube stop
# minikube start
#
# and try "kubectl top nodes" again and the Metrics Server should be working now.
#
# See also:
# * https://github.com/kubernetes-sigs/metrics-server/issues/196
# * https://github.com/juan-vg/metrics-server/commit/264f9c551abdeedc575d36e74da40671c51cb747
#
command:
- /metrics-server
- --kubelet-insecure-tls
- --kubelet-preferred-address-types=InternalIP
#
# Changes end.
#
livenessProbe:
failureThreshold: 3
httpGet:
path: /livez
port: https
scheme: HTTPS
periodSeconds: 10
name: metrics-server
ports:
- containerPort: 4443
name: https
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /readyz
port: https
scheme: HTTPS
periodSeconds: 10
securityContext:
readOnlyRootFilesystem: true
runAsNonRoot: true
runAsUser: 1000
volumeMounts:
- mountPath: /tmp
name: tmp-dir
nodeSelector:
kubernetes.io/os: linux
priorityClassName: system-cluster-critical
serviceAccountName: metrics-server
volumes:
- emptyDir: {}
name: tmp-dir
---
apiVersion: apiregistration.k8s.io/v1
kind: APIService
metadata:
labels:
k8s-app: metrics-server
name: v1beta1.metrics.k8s.io
spec:
group: metrics.k8s.io
groupPriorityMinimum: 100
insecureSkipTLSVerify: true
service:
name: metrics-server
namespace: kube-system
version: v1beta1
versionPriority: 100
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment