GKE Dataplane V2 Cilium Monitoring

README.md

Snippets to monitor Cilium's metrics on GKE Dataplane V2 with Prometheus + Grafana

Limitations

Please note that L7 Hubble metrics such as http and dns requires Cilium L7 Policy enabled, but GKE Dataplane V2 doesn't support it.

Prometheus Metrics

1. Edit Configmap kubectl edit configmap -n kube-system cilium-config to add these lines :

hubble-metrics: "flow drop icmp port-distribution tcp"
hubble-metrics-server: ":9091"

2. Edit Daemonset kubectl edit daemonset -n kube-system anetd to add the port:

...
ports:
- containerPort: 9091
  hostPort: 9091
  name: hubble-metrics
  protocol: TCP

3. Deploy PodMonitors :

apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: cilium
  namespace: kube-system
  labels:
    app: cilium
spec:
  selector:
    matchLabels:
      k8s-app: cilium
  podMetricsEndpoints:
    - port: advdpmetrics
    - port: hubble-metrics
---
apiVersion: monitoring.coreos.com/v1
kind: PodMonitor
metadata:
  name: cilium-operator
  namespace: kube-system
  labels:
    app: cilium-operator
spec:
  selector:
    matchLabels:
      io.cilium/app: operator
      name: cilium-operator
  podMetricsEndpoints:
    - port: advdpopmetrics

Grafana Dashboards

You can use the official Grafana dashboards availaible here : https://grafana.com/orgs/cilium/dashboards

Hubble UI

Check this repo : https://github.com/rueian/gke-hubble-export