Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
How to run go pprof on Docker daemon

#Use pprof debug docker daemon

pprof debug entrypoint

source file: docker\api\server\profiler.go

const debugPathPrefix = "/debug/"

func profilerSetup(mainRouter *mux.Router) {
	var r = mainRouter.PathPrefix(debugPathPrefix).Subrouter()
	r.HandleFunc("/vars", expVars)
	r.HandleFunc("/pprof/", pprof.Index)
	r.HandleFunc("/pprof/cmdline", pprof.Cmdline)
	r.HandleFunc("/pprof/profile", pprof.Profile)
	r.HandleFunc("/pprof/symbol", pprof.Symbol)
	r.HandleFunc("/pprof/trace", pprof.Trace)
	r.HandleFunc("/pprof/block", pprof.Handler("block").ServeHTTP)
	r.HandleFunc("/pprof/heap", pprof.Handler("heap").ServeHTTP)
	r.HandleFunc("/pprof/goroutine", pprof.Handler("goroutine").ServeHTTP)
	r.HandleFunc("/pprof/threadcreate", pprof.Handler("threadcreate").ServeHTTP)
}

Start docker daemon in debug mode

add -D to command line

$ /usr/bin/docker daemon -D -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock

Run socat to make docker sock available via tcp port

note the IP to listen at

$ socat -d -d TCP-LISTEN:8080,fork,bind=192.168.1.137 UNIX:/var/run/docker.sock
  2016/01/23 12:59:43 socat[3113] N listening on AF=2 192.168.1.137:8080

Access debug url entrypoint

show global vars

$ curl -s http://192.168.1.137:8080/debug/vars | jq .
  {
    "cmdline": [
      "/usr/bin/docker",
      "daemon",
      "-D",
      "-H",
      "tcp://0.0.0.0:2375",
      "-H",
      "unix:///var/run/docker.sock",
      "-api-enable-cors",
      "--storage-driver=aufs",
      "--insecure-registry",
      "registry.hyper.sh:5000",
      "--insecure-registry",
      "192.168.1.137:5000"
    ],
    "memstats": {
      "Alloc": 7140904,
      "TotalAlloc": 20379824,
      "Sys": 14559480,
      "Lookups": 3090,
      "Mallocs": 313164,
      "Frees": 199260,
      "HeapAlloc": 7140904,
  ...

get command line

$ curl -s http://192.168.1.137:8080/debug/pprof/cmdline
  /usr/bin/docker daemon -D -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock -api-enable-cors --storage-driver=aufs --insecure-registry registry.hyper.sh:5000 --insecure-registry 192.168.1.137:5000

run pprof on your client:

$ go tool pprof http://192.168.1.137:8080/debug/pprof/profile
  Fetching profile from http://192.168.1.137:8080/debug/pprof/profile
  Please wait... (30s)
  Saved profile in /home/xjimmy/pprof/pprof.192.168.1.137:8080.samples.cpu.001.pb.gz
  Entering interactive mode (type "help" for commands)
  (pprof) web     
  (pprof) web ploop
  (pprof)

generate graph

$ cd /home/xjimmy/pprof
$ gunzip pprof.192.168.1.137:8080.samples.cpu.001.pb.gz
$ go tool pprof --pdf pprof.dockerd.192.168.1.137:8080.samples.cpu.001.pb > callgraph.pdf

get symbol

$ curl -s http://192.168.1.137:8080/debug/pprof/symbol
  num_symbols: 1

other

$ culr -s http://192.168.1.137:8080/debug/pprof/block
$ curl -s http://192.168.1.137:8080/debug/pprof/heap
$ curl -s http://192.168.1.137:8080/debug/pprof/goroutine
$ curl -s http://192.168.1.137:8080/debug/pprof/threadcreate
@Bablzz

This comment has been minimized.

Copy link

Bablzz commented Sep 30, 2019

Thanks for a good article

Typo in other section
culr

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.