Using go bench, to benchmark Continuum's Policy Engine

peBench.md

Methodology

• I used my laptop to benchmark the policy object.
• I wrote a Bench function that fulfilled the Benchmark signature, each loading different numbers and types of rules.

  func BenchmarkLoadFlat(b *testing.B) {
  	benchLoad(b, "testFlat", SYNTH_RULE_FLAT, 100, 10)
  }

  func BenchmarkLoadChained(b *testing.B) {
	benchLoad(b, "testChained", SYNTH_RULE_CHAINED, 100, 10)
  }

  func BenchmarkLoadBushy(b *testing.B) {
	benchLoad(b, "testBushy", SYNTH_RULE_BUSHY, 100, 10)
  }

• Use go test bench framework to determine the execution time for the component example:

  $ go test test.bench .
  ..
  $ go test test.bench LoadRules
PASS
BenchmarkLoadRules10 After Add: rules=100 Seals=0
After Add: rules=100 Seals=0
After Add: rules=100 Seals=0
After Add: rules=100 Seals=0
500000 3398 ns/op
...

This gives me the number of nanoseconds per operation on the Benchmark I wrote. I put the results into a spreadsheet and graphed them. Then for any problems identified I built a profile. This was run from the Ubuntu Vagrant test on my laptop. I used Ubuntu because OSX did not properly collect profile data.

$ go test -test.bench EvalChainRules -test.cpuprofile=prof_EvalChainRules.out 
PASS
BenchmarkEvalChainRules10 After Add: rules=100 Seals=0
After Add: rules=100 Seals=0
After Add: rules=100 Seals=0
After Add: rules=100 Seals=0
200000 10807 ns/op
BenchmarkEvalChainRules100After Add: rules=1000 Seals=0
After Add: rules=1000 Seals=0
After Add: rules=1000 Seals=0
After Add: rules=1000 Seals=0
50000 59505 ns/op
BenchmarkEvalChainRules500After Add: rules=5000 Seals=0
After Add: rules=5000 Seals=0
After Add: rules=5000 Seals=0
...

• I copy the pengine.test and prof*.out files to osx.
• Then finally on osx I just execute go tool pprof with --web option. This starts up the browser with the profile graph.

go tool pprof --web pengine.test prof_EvalChainRules.out

• References

http://golang.org/pkg/runtime/pprof/
https://www.datadoghq.com/2014/04/go-performance-tales/
http://saml.rilspace.org/profiling-and-creating-call-graphs-for-go-programs-with-go-tool-pprof