Monitoring External Etcd Cluster With Prometheus Operator

monitor-external-etcd-cluster-with-p8s-operator.md

Monitoring External Etcd Cluster With Prometheus Operator

Credits

• https://jpweber.io/blog/monitoring-external-etcd-cluster-with-prometheus-operator/
• https://github.com/xetys/rancher-prometheus/blob/master/docs/Monitoring%20external%20etcd.md

Create etcd-certs secret

$ kubectl create secret generic etcd-certs \
    -n monitoring \
    --from-file=etcd-client.pem=etcd-client.pem \
    --from-file=etcd-client-key.pem=etcd-client-key.pem \
    --from-file=ca.pem=ca.pem

Create etcd service and add endpoints

# etcd-service-and-endpoints.yml
apiVersion: v1
kind: Service
metadata:
  name: etcd
  namespace: kube-system
  labels:
    k8s-app: etcd
spec:
  clusterIP: None
  ports:
  - name: metrics
    port: 2379
    targetPort: 2379
  selector: null
---
apiVersion: v1
kind: Endpoints
metadata:
  name: etcd
  namespace: kube-system
  labels:
    k8s-app: etcd
subsets:
- addresses:
  - ip: 10.44.15.221
  - ip: 10.44.15.222
  - ip: 10.44.15.223
  ports:
  - name: metrics
    port: 2379
    protocol: TCP

$ kubectl apply -f etcd-service-and-endpoints.yml

Prometheus-Operator Helm Chart

Add secret to p8s-operator helm chart

prometheus:
  prometheusSpec:
    secrets: 
      - etcd-certs

Enable etcd service monitor

# prometheus-operator helm chart
kubeEtcd:
  enabled: true

  serviceMonitor:
    scheme: https
    insecureSkipVerify: false
    caFile: /etc/prometheus/secrets/etcd-certs/ca.pem
    certFile: /etc/prometheus/secrets/etcd-certs/etcd-client.pem
    keyFile: /etc/prometheus/secrets/etcd-certs/etcd-client-key.pem
    selector:
      matchLabels:
        k8s-app: etcd

$ helm upgrade --install stable/prometheus-operator -n monitoring -f values.yml