Skip to content

Instantly share code, notes, and snippets.

@nmasse-itix
Created July 5, 2018 17:08
Show Gist options
  • Save nmasse-itix/0173f925587155ce600a74fe6a912595 to your computer and use it in GitHub Desktop.
Save nmasse-itix/0173f925587155ce600a74fe6a912595 to your computer and use it in GitHub Desktop.
How to debug issues in OpenShift

How to debug issues in OpenShift

Context

Lets say that you deployed an application in OpenShift and the application is not working. You would like to debug but the application does not embed any troubleshooting tool (for instance, an Alpine Linux or a scratch image)

Static tools

You could download one of the static tools available there in /tmp and run it from there.

Sidecar container

For instance, if you need to troubleshoot network issues:

Deploy our boggus application

oc new-app --name boggus alpine:latest
oc patch dc boggus --type=json -p '[{"op": "add", "path": "/spec/template/spec/containers/0/command", "value": ["/bin/sh", "-c", "while :; do sleep 1; done" ]}]'

Add a sidecar container that has the tools to debug network issues

oc patch dc boggus --type=json -p '[{"op": "add", "path": "/spec/template/spec/containers/1", "value": { "image": "szalek/pentest-tools", "name": "debug", "command": [ "/bin/sh", "-c", "while :; do sleep 1; done" ]} }]'

Enter the sidecar container

oc rsh -c debug $(oc get pods -l app=boggus -o name|tail -n 1)

For strace, it is a bit more complicated since you will have access to the host PID namespace.

Give privileged rights to the default service account

oc adm policy add-scc-to-user privileged -z default

Add a sidecar container that has strace

oc patch dc boggus --type=json -p '[{"op": "add", "path": "/spec/template/spec/containers/1", "value": { "image": "benhall/strace-ubuntu", "name": "debug", "command": [ "/bin/sh", "-c", "while :; do sleep 1; done" ], "securityContext": { "privileged": true } } }, {"op": "add", "path": "/spec/template/spec/hostPID", "value": true } ]'

Enter the sidecar container

oc rsh -c debug $(oc get pods -l app=boggus -o name|tail -n 1)

In the container, try:

ps ax

and then:

strace -ff -p <pid>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment