Skip to content

Instantly share code, notes, and snippets.

@stevenctl
Last active January 20, 2023 20:07
Show Gist options
  • Star 2 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save stevenctl/938a48edf645dab13deec93ded626e1b to your computer and use it in GitHub Desktop.
Save stevenctl/938a48edf645dab13deec93ded626e1b to your computer and use it in GitHub Desktop.
Debugging Pilot in K8S

Debugging Pilot with Delve

First build pilot with GCFLAGS="all=-n -l":

GCFLAGS="all=-N -l" HUB=docker.io/slandow TAG=slandow BUILD_IN_CONTAINER=1 make docker.pilot

Then we can wrap that image in one that includes delve:

FROM golang:1.14.1-alpine AS build-env

ENV CGO_ENABLED 0

# Allow Go to retreive the dependencies for the build step
RUN apk add --no-cache git
RUN go get github.com/go-delve/delve/cmd/dlv

FROM slandow/pilot:slandow
COPY --from=build-env /go/bin/dlv /

EXPOSE 40000

ENTRYPOINT ["/dlv", "--listen=:40000", "--headless=true", "--api-version=2", "--accept-multiclient", "exec", "/usr/local/bin/pilot-discovery", "--" ]
# Warning, this will overwrite your normal pilot image.
# If you just want to build it and save it for later, use a different tag. 
# You may be able to use this process for all of the images, except use TAG=delve instead. 
docker build -f Dockerfile.delve -t slandow/pilot:slandow .
docker push slandow/pilot:slandow

Install istio, specifying the values for hub and tag:

istioctl manifest apply \
  --set values.global.hub=$HUB \
  --set values.global.tag=$TAG

While the installation waits on - Processing resources for components Pilot. Waiting for Deployment/istio-system/istiod you can move on to the next two steps. Delve is waiting for a bit to let a debugger attach to the very start of the program.

Port forward the istiod pod:

kubectl --context=$CTX_1 -n istio-system port-forward $(kubectl --context=$CTX_1 -n istio-system get pod -l app=istiod -o jsonpath='{.items[0].metadata.name}') 40000:40000

Attach your GoLand to the remote session. In GoLand go to Run > Edit Configurations > "+" > Go Remote and enter 127.0.0.1:40000

@stevenctl
Copy link
Author

stevenctl commented May 13, 2020

@howardjohn have you tried anything like this/is there an existing remote debugging solution? If not I may roll it into a make target all the containers and tag them <user>/<container>:delve

@howardjohn
Copy link

@howardjohn
Copy link

There is also https://github.com/solo-io/squash but i haven't tried it either

@rinormaloku
Copy link

Thanks for the guide Steven! Had to add the variable DEBUG=1, so that the binary is not stripped of debugging info:

GCFLAGS="all=-N -l" DEBUG=1 HUB=docker.io/slandow TAG=slandow BUILD_IN_CONTAINER=1 make docker.pilot

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment