Delete evicted pods

gistfile1.txt

kubectl get pods | grep Evicted | awk '{print $1}' | xargs kubectl delete pod

It happen with me when I installed Kind Cluster alongside with docker-for-desktop cluster. This is how I resolve it

kubectl get po --all-namespaces --field-selector 'status.phase==Failed' -o json | kubectl delete -f -

kubectl get po --all-namespaces --field-selector 'status.phase==Evicted' -o json | kubectl delete -f -

kubectl get po --all-namespaces --field-selector 'status.phase==Pending' -o json | kubectl delete -f -

For me, kubectl delete pod needs the -n cattle-system to work!
kubectl get pods --all-namespaces | grep Evicted | awk '{print $2}' | xargs kubectl -n cattle-system delete pod

kubectl get po -A | grep Evicted | awk '{system ("kubectl -n " $1 " delete po " $2)}'

why not kubectl delete pod --all-namespaces --field-selector 'status.phase==Evicted'?

@vdksystem the Evictedis under reason path and field-selector by status.reason isn't allowed. We can do this: kubectl delete pod --all-namespaces --field-selector 'status.phase=Failed'

Sorry, the question is why not kubectl delete [selector]
Looks more elegant than kubectl get pods [selector] -o json | kubectl delete -f -

I like the command from @bronweg above using the field selectors option.

Except maybe to be safer, you can explicitly look for "Evicted" pods instead of ones who status is not "Running". I also recommend doing a kubectl get pods w/the same options to see what you're going to delete before actually deleting them

kubectl get pods --field-selector=status.phase=Evicted
kubectl delete pods --field-selector=status.phase=Evicted

While the status.phase can be running, I did not find any "Evicted" ones even though they were actually that. I had better luck using a lookup for "Failed" when it is "Evicted", i.e. :

kubectl get pods --field-selector=status.phase=Failed
kubectl delete pods --field-selector=status.phase=Failed

@rschoultz exactly! This is what I mean with my comment above. Evicted is under status.reason and this one isn't available to use as field-selector, e.g.:

#: kubectl get pods --field-selector=status.reason=Evicted
Error from server (BadRequest): Unable to find "/v1, Resource=pods" that match label selector "", field selector "status.reason=Evicted": field label not supported: status.reason

status:
  message: ""
  phase: Failed
  reason: Evicted
  startTime: ""

is there a way to remove old evicted pods?

kubectl delete pods --field-selector=status.phase!=Running -A ?

Another version:
kubectl delete pod $(kubectl get pods | awk '/Evicted/ {print $1}')

One to delete on a specific namespace:
NAMESPACE=jenkins sh -c 'for pod in $(kubectl get pods -n "$NAMESPACE"|grep Evicted|awk "{print $1}"); do kubectl delete pod -n "$NAMESPACE" $pod; done'

kubectl get po -A --all-namespaces -o json | jq  '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete po \(.metadata.name) -n \(.metadata.namespace)"' | xargs -n 1 bash -c

kubectl get po -A --all-namespaces -o json | jq  '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete po \(.metadata.name) -n \(.metadata.namespace)"' | xargs -n 1 bash -c

Thank you!

kubectl get po -A |grep Evicted|awk '{print "kubectl delete po -n ",$1,$2}'|bash -x

kubectl delete pod $(kubectl get pods | grep Evicted | awk '{print $1}')

kubectl delete pod $(kubectl get pods | grep Evicted | awk '{print $1}')

not have namespace,It's just for default namespace ,sad for you.

kubectl delete pod $(kubectl get pods | grep Evicted | awk '{print $1}')

not have namespace,It's just for default namespace ,sad for you.

How hard is it to include the namespace with the flag -n

And using just kubectl kubectl delete pod $(kubectl get pod -o jsonpath="{range .items[*]}{.metadata.name}{\" \"}{end}" --field-selector="status.phase==Failed")

How to reproduce this command for powershell?

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

Your answer worth the scroll :)) Should be the top answer

Everybody who's a little concerned about running delete commands can use this the verify which pods would be affected:

kubectl get pod -A --field-selector="status.phase==Failed"

For kubectl get and kubectl delete you can use -A if you want to run it for all namespaces, otherwise a -n NAMESPACE to limit the scope to one specific namespace.

like @qinghon:
kubectl get po -A | awk '/Evicted|Completed/ {print "kubectl delete po -n ",$1,$2}'|bash -x
The gap about this approach without grep is that you filter for the status after getting all the input

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

💯

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

👍

Just created a handy function. You can add this to your .zshrc or .bash_profile and it will just clean everything for you. Just add your namespaces to the array.

function deleteEvictedPods() {
    environments=(
        development
        staging
        production
        traefik
        kube-system
        kube-public
        kube-node-lease
        default
    )

    for environment in "${environments[@]}"
    do
        echo "==================================================================================="
        echo "Removing evicted pods for namespace: \"${environment}\""
        echo "---------------------------------------------------"

        for each in $(kubectl get pods -n ${environment} | grep Evicted | awk '{print $1}');
        do
            kubectl delete pods $each -n ${environment}
        done
        echo "==================================================================================="
        echo "\n"
    done
}

output

===================================================================================
===================================================================================
Removing evicted pods for namespace: "development"
---------------------------------------------------
pod "example-frontend-primary-757586fc7c-2zrp8" deleted
pod "example-frontend-primary-757586fc7c-54k6h" deleted
pod "example-frontend-primary-757586fc7c-62g5n" deleted
pod "example-frontend-primary-757586fc7c-6v4kv" deleted
pod "example-frontend-primary-757586fc7c-7jh6q" deleted
pod "example-frontend-primary-757586fc7c-89bvw" deleted
pod "example-frontend-primary-757586fc7c-c7mvt" deleted
pod "example-frontend-primary-757586fc7c-cc5lt" deleted
pod "example-frontend-primary-757586fc7c-jhmm6" deleted
pod "example-frontend-primary-757586fc7c-k5xfn" deleted
pod "example-frontend-primary-757586fc7c-pdfgk" deleted
pod "example-frontend-primary-757586fc7c-qhjnx" deleted
pod "example-frontend-primary-757586fc7c-r77hj" deleted
pod "example-frontend-primary-757586fc7c-v48h2" deleted
pod "example-service-feature-flags-55f8fb4858-4hmq6" deleted
pod "example-service-feature-flags-55f8fb4858-5vk44" deleted
pod "example-service-feature-flags-55f8fb4858-mkrx9" deleted
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "staging"
---------------------------------------------------
No resources found in staging namespace.
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "production"
---------------------------------------------------
No resources found in production namespace.
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "traefik"
---------------------------------------------------
pod "example-traefik-ingress-controller-df6f59b47-8k6qx" deleted
pod "example-traefik-ingress-controller-df6f59b47-kfjbr" deleted
pod "example-traefik-ingress-controller-df6f59b47-m59s4" deleted
pod "example-traefik-ingress-controller-df6f59b47-mrq8p" deleted
pod "example-traefik-ingress-controller-df6f59b47-nbsjc" deleted
pod "example-traefik-ingress-controller-df6f59b47-pfqbg" deleted
pod "example-traefik-ingress-controller-df6f59b47-vsnqw" deleted
pod "example-traefik-ingress-controller-df6f59b47-wzd6g" deleted
pod "example-traefik-ingress-controller-df6f59b47-xpq9x" deleted
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "kube-system"
---------------------------------------------------
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "kube-public"
---------------------------------------------------
No resources found in kube-public namespace.
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "kube-node-lease"
---------------------------------------------------
No resources found in kube-node-lease namespace.
===================================================================================
===================================================================================

===================================================================================
===================================================================================
Removing evicted pods for namespace: "default"
---------------------------------------------------
No resources found in default namespace.
===================================================================================
===================================================================================

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

Thanks

kubectl get pods --all-namespaces -o json | jq '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete pods \(.metadata.name) -n \(.metadata.namespace)"' | xargs -n 1 bash -c

i think the most simple command is
kubectl delete pods -A --field-selector=status.phase=Failed

This works for me

kubectl delete pods $(kubectl get pods | grep [pod name] | grep Evicted | awk '{print $1}')

Why doesn't Kubernetes clean up Evicted pods by itself? I only notice it happen sometimes.

kubectl get pod -A | grep Evicted | awk '{print $2 " --namespace=" $1}' | xargs -n 2 kubectl delete pod

i think the most simple command is
kubectl delete pods -A --field-selector=status.phase=Failed

best solution! 👍

i think the most simple command is kubectl delete pods -A --field-selector=status.phase=Failed

works for me

kgpa | grep Evicted | awk '{print $2}' | xargs kubectl delete pod --force

kgpa | grep -v Running | awk '{print $2}' | xargs kubectl delete pod --force

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

The original question is about to delete "Evicted" pods, which is a subset of "Failed". Unfortunately there is no state field for pods that are in the Running state.

kubectl get po -A --all-namespaces -o json | jq  '.items[] | select(.status.reason!=null) | select(.status.reason | contains("Evicted")) | "kubectl delete po \(.metadata.name) -n \(.metadata.namespace)"' | xargs -n 1 bash -c

Thank you!

i think the most simple command is kubectl delete pods -A --field-selector=status.phase=Failed

Works for me too, for deleting failed pods in all namespaces

Did you know that you can use the --field-selector option for kubectl delete as well?

kubectl delete pod --field-selector="status.phase==Failed"

Great answer, thanks!

Why doesn't Kubernetes clean up Evicted pods by itself? I only notice it happen sometimes.

My understanding is that there is a threshold for cleaning up - when the number of failed hits that threshold then clean up will happen - the challenge is that the default for that is 12500 (twelve thousand five hundred). The purpose of the threshold is to allow for review of the reasons for failure and I can see that in a large system that might almost be a reasonable number.

That threshold can be changed - I'm not sure what a sensible number would look like for a small cluster.