failing on TestBenchmark laptop

jbean@Jeffreys-MacBook-Pro ~/go/src/github.com/yarpc/yab [logging]
± % ulimit -n                                                                                                                                                                      !5767
1024

jbean@Jeffreys-MacBook-Pro ~/go/src/github.com/yarpc/yab [logging *]
± % make test                                                                                                                                                                      !5751
go test -cover -race ./encoding/... ./internal/... ./limiter/... ./ratelimit/... ./sorted/... ./statsd/... ./testdata/... ./thrift/... ./transport/... ./unmarshal/... .
warning: "./testdata/..." matched no packages
ok  	github.com/yarpc/yab/encoding	1.087s	coverage: 96.8% of statements
?   	github.com/yarpc/yab/internal/thrifttest	[no test files]
ok  	github.com/yarpc/yab/limiter	1.200s	coverage: 100.0% of statements
?   	github.com/yarpc/yab/ratelimit	[no test files]
ok  	github.com/yarpc/yab/sorted	1.038s	coverage: 100.0% of statements
ok  	github.com/yarpc/yab/statsd	1.042s	coverage: 100.0% of statements
ok  	github.com/yarpc/yab/thrift	1.039s	coverage: 96.8% of statements
ok  	github.com/yarpc/yab/transport	1.161s	coverage: 89.0% of statements
ok  	github.com/yarpc/yab/unmarshal	1.032s	coverage: 100.0% of statements
--- FAIL: TestBenchmark (3.17s)
        Error Trace:    benchmark_test.go:109
	Error:  	Should be true
	Messages:	Capped by RPS * duration: Took 1.085629149s, wanted duration 500ms
		
        Error Trace:    benchmark_test.go:109
	Error:  	Should be true
	Messages:	Capped by duration: Took 1.275731981s, wanted duration 500ms
		
{
  "body": {},
  "ok": true,
  "trace": "0"
}

{
  "body": {},
  "headers": {
    "header": "values"
  },
  "ok": true,
  "trace": "0"
}

{
  "body": {
    "result": {
      "ok": true
    }
  },
  "ok": true,
  "trace": "0"
}

{
  "body": {
    "result": {
      "ok": true
    }
  },
  "ok": true,
  "trace": "0"
}

Benchmark parameters:
  CPUs:            4
  Connections:     8
  Concurrency:     1
  Max requests:    100
  Max duration:    5s
  Max RPS:         0
Latencies:
  0.5000: 7.284742ms
  0.9000: 12.861982ms
  0.9500: 20.158536ms
  0.9900: 23.745949ms
  0.9990: 23.874553ms
  0.9995: 23.881698ms
  1.0000: 23.888843ms
Elapsed time:      108ms
Total requests:    100
RPS:               923.10
{
  "body": {
    "result": {
      "ok": true
    }
  },
  "ok": true,
  "trace": "0"
}

Benchmark parameters:
  CPUs:            4
  Connections:     3
  Concurrency:     1
  Max requests:    0
  Max duration:    100ms
  Max RPS:         1
Latencies:
  0.5000: 5.474976ms
  0.9000: 6.472526ms
  0.9500: 6.551663ms
  0.9900: 6.614972ms
  0.9990: 6.629217ms
  0.9995: 6.630008ms
  1.0000: 6.6308ms
Elapsed time:      109ms
Total requests:    4
RPS:               36.42
{
  "body": {},
  "headers": {
    "header1": "value1",
    "header2": "value2"
  },
  "ok": true,
  "trace": "0"
}

FAIL
coverage: 95.1% of statements
FAIL	github.com/yarpc/yab	4.621s
make: *** [test] Error 1

Looking at the actual time for the tests we determined it was the warmup connections skewing the duration.

Benchmark parameters:
  CPUs:            4
  Connections:     50
  Concurrency:     2
  Max requests:    100
  Max duration:    1m40s
  Max RPS:         0
Latencies:
  0.5000: 39.378767ms
  0.9000: 56.639468ms
  0.9500: 62.076522ms
  0.9900: 65.304235ms
  0.9990: 65.583276ms
  0.9995: 65.598778ms
  1.0000: 65.614281ms
Elapsed time:      86ms
Total requests:    100
RPS:               1161.44

Benchmark parameters:
  CPUs:            4
  Connections:     50
  Concurrency:     2
  Max requests:    60
  Max duration:    500ms
  Max RPS:         120
Latencies:
  0.5000: 2.703599ms
  0.9000: 3.034549ms
  0.9500: 3.291331ms
  0.9900: 6.121968ms
  0.9990: 9.718113ms
  0.9995: 9.917899ms
  1.0000: 10.117685ms
Elapsed time:      505ms
Total requests:    60
RPS:               118.65

Benchmark parameters:
  CPUs:            4
  Connections:     50
  Concurrency:     2
  Max requests:    0
  Max duration:    500ms
  Max RPS:         0
Latencies:
  0.5000: 96.121945ms
  0.9000: 164.777916ms
  0.9500: 177.960542ms
  0.9900: 197.645767ms
  0.9990: 212.218986ms
  0.9995: 213.191323ms
  1.0000: 214.16366ms
Elapsed time:      577ms
Total requests:    524
RPS:               907.44