Gist to embed PrometheusRules resource usage (PVC only) (as of a7e23f4c05b5c26489ee3ca926287f9b087a1582).

blog.content.post.reducing-the-cost-of-running-a-personal-k8s-cluster-part-2.rules-resource-usage-pvc-only.yaml

---
# This manifest defines the PrometheusRules specific to resource usage
# for monitoring/alerting our k8s cluster.
#
# - `severity=critical` indicates we should raise an alert via Opsgenie.
# - 15m is the default `for`. We may refine this value over time.
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
  name: resource-usage
  labels:
    app: resource-usage
    # Prometheus searches for any PrometheusRules with the `role=alert-rules`
    # label.
    role: alert-rules
spec:
  groups:
  - name: resource-usage.rules
    rules:
      ...
      - alert: PersistentVolumeClaimLowAvailableSpace
        annotations:
          message: 'A persistent volume claim is almost out of available space.'
        expr: 'count((min by(persistentvolumeclaim)(kubelet_volume_stats_available_bytes) / min by(persistentvolumeclaim)(kubelet_volume_stats_capacity_bytes)) < .1) >= 1'
        for: 15m
        labels:
          severity: critical
      ...