Skip to content

Instantly share code, notes, and snippets.

@vfarcic
Last active November 28, 2023 22:36
Show Gist options
  • Star 1 You must be signed in to star a gist
  • Fork 1 You must be signed in to fork a gist
  • Save vfarcic/e593ff9fa1a34d286e4b7f739bf47bc2 to your computer and use it in GitHub Desktop.
Save vfarcic/e593ff9fa1a34d286e4b7f739bf47bc2 to your computer and use it in GitHub Desktop.
# Source: https://gist.github.com/e593ff9fa1a34d286e4b7f739bf47bc2
###################################################
# Monitoring, Logging, And Alerting In Kubernetes #
# https://youtu.be/XR_yWlOEGiA #
###################################################
# Additional Info:
# - Prometheus: https://prometheus.io
# - Robusta: https://robusta.dev
# - Loki: https://grafana.com/oss/loki
# - Grafana: https://grafana.com/oss/grafana
# - Kubernetes Notifications, Troubleshooting, And Automation With Robusta: https://youtu.be/2P76WVVua8w
#########
# Setup #
#########
# Create a Kubernetes cluster with Ingress
# Using Rancher Desktop for the demo, but it can be any other Kubernetes cluster with Ingress
# If not using Rancher Desktop, replace `127.0.0.1` with the base host accessible through NGINX Ingress
export INGRESS_HOST=127.0.0.1
# # Change the value if NOT using NGINX Ingress
# export INGRESS_CLASS_NAME=nginx
helm repo add prometheus \
https://prometheus-community.github.io/helm-charts
helm repo add grafana \
https://grafana.github.io/helm-charts
helm repo update
git clone https://github.com/vfarcic/prometheus-loki-grafana-demo
cd prometheus-loki-grafana-demo
########################################
# Metrics And Alerting With Prometheus #
########################################
# If Ingress is not set to act as default, you might need to add `--set server.ingress.ingressClassName=YOUR_INGRESS`.
helm upgrade --install \
prometheus prometheus/prometheus \
--namespace monitoring \
--create-namespace \
--values prometheus-values.yaml \
--set "server.ingress.hosts[0]=prometheus.$INGRESS_HOST.nip.io" \
--wait
echo "http://prometheus.$INGRESS_HOST.nip.io"
# Open the output in a browser
# container_cpu_usage_seconds_total
# rate(container_cpu_usage_seconds_total[5m])
# Open https://prometheus.io/docs/prometheus/latest/querying/functions/
# rate(container_cpu_usage_seconds_total{namespace="monitoring"}[5m])
# rate(container_cpu_usage_seconds_total{namespace="monitoring", container="prometheus-node-exporter"}[5m])
# Open https://github.com/prometheus-community/helm-charts/tree/main/charts
#############################
# Logs Collection With Loki #
#############################
helm upgrade --install \
loki grafana/loki-stack \
--namespace monitoring \
--create-namespace \
--wait
###########################
# Dashboards With Grafana #
###########################
cat grafana-values.yaml
# Open https://grafana.com/grafana/dashboards in a browser
helm upgrade --install \
grafana grafana/grafana \
--namespace monitoring \
--create-namespace \
--values grafana-values.yaml \
--set "ingress.hosts[0]=grafana.$INGRESS_HOST.nip.io" \
--wait
kubectl --namespace monitoring \
get secret grafana \
--output jsonpath="{.data.admin-password}" \
| base64 --decode ; echo
echo "http://grafana.$INGRESS_HOST.nip.io"
# User: `admin`
# Password: the output of the `echo` command
# rate(container_cpu_usage_seconds_total{namespace="monitoring", container="prometheus-node-exporter"}[5m])
###########
# Destroy #
###########
# Destroy or reset the cluster
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment