Skip to content

Instantly share code, notes, and snippets.

@ElanHasson
Last active April 9, 2024 03:01
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ElanHasson/f3305f6108e386bc180a273caa434e5d to your computer and use it in GitHub Desktop.
Save ElanHasson/f3305f6108e386bc180a273caa434e5d to your computer and use it in GitHub Desktop.
This script deletes all PVCs and restarted degraded pods.
#!/bin/bash
# Get all PVs in the 'Released' state in JSON format
RELEASED_PVS_JSON=$(kubectl get pv -o json | jq -r '.items[] | select(.status.phase == "Released")')
# Check if there are no 'Released' PVs
if [ -z "$RELEASED_PVS_JSON" ]; then
echo "No PVs in 'Released' state found."
exit 0
fi
echo "Processing PVs in 'Released' state for forced deletion..."
# Loop through each 'Released' PV and patch it to remove finalizers, then delete
echo "$RELEASED_PVS_JSON" | jq -r '.metadata.name' | while read -r PV_NAME; do
echo "Removing finalizers for PV: $PV_NAME..."
kubectl patch pv "$PV_NAME" -p '{"metadata":{"finalizers":[]}}' --type=merge
echo "Forcing deletion of PV: $PV_NAME..."
kubectl delete pv "$PV_NAME" --force --grace-period=0
echo "Deleted PV: $PV_NAME"
done
echo "All 'Released' PVs have been processed for forced deletion."
#!/bin/bash
restart_pod() {
local pod_name=$1
local namespace=$2
echo "Restarting pod $pod_name in $namespace..."
# Note: The pod will be recreated automatically if it is managed by a Deployment, StatefulSet, etc.
kubectl delete pod $pod_name --namespace $namespace
}
echo "Checking for degraded pods across all namespaces..."
DEGRADED_POD_INFO=$(kubectl get pods --all-namespaces --field-selector=status.phase!=Running,status.phase!=Succeeded -o jsonpath="{range .items[*]}{.metadata.namespace}{'\t'}{.metadata.name}{'\n'}{end}")
while IFS= read -r line; do
NAMESPACE=$(echo "$line" | cut -f1)
POD=$(echo "$line" | cut -f2)
PVCS=$(kubectl get pod $POD --namespace $NAMESPACE -o jsonpath="{.spec.volumes[*].persistentVolumeClaim.claimName}")
for PVC in $PVCS; do
if [ ! -z "$PVC" ]; then
echo "Namespace: $NAMESPACE, Degraded Pod: $POD is using PVC: $PVC"
echo "Deleting PVC $PVC in $NAMESPACE..."
kubectl delete pvc "$PVC" --namespace "$NAMESPACE"
# Restart the pod to pick up the new PVC. Caution: this can result in downtime.
restart_pod "$POD" "$NAMESPACE"
fi
done
done <<< "$DEGRADED_POD_INFO"
echo "Operation completed."
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment