Skip to content

Instantly share code, notes, and snippets.

@vfarcic
Last active January 24, 2023 23:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vfarcic/e9ff9ba54c4bd4b2d38349262a20935c to your computer and use it in GitHub Desktop.
Save vfarcic/e9ff9ba54c4bd4b2d38349262a20935c to your computer and use it in GitHub Desktop.
# Source: https://gist.github.com/e9ff9ba54c4bd4b2d38349262a20935c
################################################################
# Postmortem Documents About Production Issues With Fiberplane #
# https://youtu.be/sLV6zCRlDEg #
################################################################
# Additional Info:
# - Fiberplane: https://fiberplane.dev
# - Monitoring, Logging, And Alerting In Kubernetes: https://youtu.be/XR_yWlOEGiA
# - You MUST Instrument Your Code With OpenTelemetry!: https://youtu.be/oe5YYh9mhzw
# - Distributed Tracing With Jaeger And OpenTelemetry In Kubernetes: https://youtu.be/FK0uh-7nDSg
# - Kubernetes Notifications, Troubleshooting, And Automation With Robusta: https://youtu.be/2P76WVVua8w
# - The Best Performance And Load Testing Tool? k6 By Grafana Labs: https://youtu.be/5OgQuVAR14I
#########
# Setup #
#########
git clone https://github.com/vfarcic/fiberplane-demo
cd fiberplane-demo
# Create a Kubernetes cluster.
# If you choose to use a local cluster (e.g., Minikube,
# Rancher Desktop, Docker Desktop, etc.), some commands
# (especially those related to retrieving the IP) might differ.
# If you prefer to follow the demo as-is, please use a remote
# cluster like GKE, EKS, AKS, etc.
helm upgrade --install ingress-nginx ingress-nginx \
--repo https://kubernetes.github.io/ingress-nginx \
--namespace ingress-nginx --create-namespace \
--values nginx-values.yaml --wait
# If NOT EKS
export INGRESS_HOST=$(kubectl --namespace ingress-nginx \
get service ingress-nginx-controller \
--output jsonpath="{.status.loadBalancer.ingress[0].ip}")
# If EKS
export INGRESS_HOSTNAME=$(kubectl --namespace ingress-nginx \
get service ingress-nginx-controller \
--output jsonpath="{.status.loadBalancer.ingress[0].hostname}")
# If EKS
export INGRESS_HOST=$(dig +short $INGRESS_HOSTNAME)
echo $INGRESS_HOST
# Repeat the `export` command(s) if the output is empty.
# If the output contains more than one IP, wait for a while longer, and repeat the `export` commands.
# If the output continues having more than one IP, choose one of them and execute `export INGRESS_HOST=[...]` with `[...]` being the selected IP.
helm upgrade --install \
prometheus prometheus \
--repo https://prometheus-community.github.io/helm-charts \
--namespace monitoring \
--create-namespace \
--set "server.ingress.hosts[0]=prometheus.$INGRESS_HOST.nip.io" \
--values prometheus-values.yaml \
--wait
# Install `yq` from https://github.com/mikefarah/yq if you do not have it already
yq --inplace \
".spec.rules[0].host = \"silly-demo.$INGRESS_HOST.nip.io\"" \
k8s/ing.yaml
cat k6.js \
| sed -e "s@http\.get.*@http\.get('http://silly-demo.$INGRESS_HOST.nip.io');@" \
| tee k6.js
cat k6-db.js \
| sed -e "s@http\.get.*@http\.get('http://silly-demo.$INGRESS_HOST.nip.io/video?id=something\&title=else');@" \
| tee k6-db.js
kubectl create namespace production
kubectl --namespace production apply --filename k8s
# Watch https://youtu.be/5OgQuVAR14I if you're not familiar with k6
curl --proto '=https' --tlsv1.2 -sSf https://fp.dev/install.sh \
| sh
fp login
fp proxies create dot
# Replace `[...]` with Token available in the output of the
# previous command.
export PROXY_API_TOKEN=[...]
yq --inplace \
".spec.template.spec.containers[0].env[0].value = \"$PROXY_API_TOKEN\"" \
fiberplane/deployment.yaml
kubectl create namespace fiberplane
kubectl --namespace fiberplane apply --filename fiberplane/
k6 run k6.js
k6 run k6-db.js
#######################
# Fiberplane Hands-On #
#######################
# Open https://studio.fiberplane.com in a browser
# Create a "New notebook"
# Check the datasources
# `/Prometheus chart`
# nginx_ingress_controller_requests
# nginx_ingress_controller_requests{status!="200"}
# sum(kube_pod_container_resource_limits{resource="cpu"}) - sum(kube_node_status_capacity{resource="cpu"})
# sum(kube_pod_container_resource_limits{resource="memory"}) - sum(kube_node_status_capacity{resource="memory"})
# min_over_time(sum by (namespace, pod) (kube_pod_status_phase{phase=~"Pending|Unknown|Failed"})[15m:1m]) > 0
###########
# Destroy #
###########
yq --inplace \
".spec.template.spec.containers[0].env[0].value = \"my-api-token\"" \
fiberplane/deployment.yaml
# Destroy or reset the Kubernetes cluster
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment