Skip to content

Instantly share code, notes, and snippets.

Last active January 31, 2023 21:03
  • Star 2 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
What would you like to do?
Detect half-open connections in Kubernetes
pods=$(kubectl get po -A -l -ojsonpath="{range .items[*]}{} {.metadata.namespace}{'\n'}{end}")
IFS=" "
while read name namespace; do
tcp=$(kubectl debug -n $namespace $name -it -- cat /proc/net/tcp)
close_wait=$(echo $tcp | awk 'BEGIN {cnt=0} $4==08 {cnt++} END {print cnt}')
fin_wait_2=$(echo $tcp | awk 'BEGIN {cnt=0} $4==05 {cnt++} END {print cnt}')
if [ "$close_wait" -gt "0" -o "$fin_wait_2" -gt "0" ]; then
echo "$name.$namespace has $close_wait sockets in CLOSE_WAIT and $fin_wait_2 sockets in FIN_WAIT_2"
echo "$name.$namespace is okay"
done <<< "$pods"
Copy link

rootik commented Sep 8, 2022

exec failed: container_linux.go:380: starting container process caused: exec: "cat": executable file not found in $PATH: unknown

Copy link

adleong commented Sep 9, 2022

Thanks, @rootik! Good catch: the script stopped working since the cat utility is no longer included in the linkerd-proxy container.

I've updated the script to instead use a debug ephemeral container.

Copy link

rootik commented Sep 9, 2022


Copy link

error: ephemeral containers are disabled for this cluster (error from server: "the server could not find the requested resource").

Server Version: version.Info{Major:"1", Minor:"22", GitVersion:"v1.22.8-gke.202", GitCommit:"88deae00580af268497b9656f216cb092b630563", GitTreeState:"clean", BuildDate:"2022-06-03T03:27:52Z", GoVersion:"go1.16.14b7", Compiler:"gc", Platform:"linux/amd64"}

Copy link

adleong commented Sep 12, 2022

@dwilliams782 yes, this requires that ephemeral container support is enabled on the cluster. otherwise you will need to add the debug containers at inject time. see:

Copy link

Cool! NP - I haven't dug into that debug container use case but it sounds like it might help with my most recent issue - link

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment