Skip to content

Instantly share code, notes, and snippets.

@lestrrat

lestrrat/gist:a2ddee74143f63c9e1c1bcbf1f4211ae

Created February 10, 2021 07:18
Show Gist options
  • Save lestrrat/a2ddee74143f63c9e1c1bcbf1f4211ae to your computer and use it in GitHub Desktop.
Save lestrrat/a2ddee74143f63c9e1c1bcbf1f4211ae to your computer and use it in GitHub Desktop.
Download ZIP
baseline-vs-v0.4.2
Raw
gistfile1.txt
lestrrat@finch bench % benchstat -sort -delta stdlib.txt v0.4.2.txt
name old time/op new time/op delta
JWK/Serialization/RSA/PrivateKey/jwk.Parse-8 50.0µs ± 1% 15.8µs ± 8% -68.40% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/jwk.ParseString-8 50.4µs ± 1% 18.5µs ±15% -63.23% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.Parse-8 15.6µs ± 1% 5.8µs ± 5% -63.11% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.ParseString-8 15.9µs ± 3% 5.9µs ± 2% -62.64% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/jwk.ParseReader-8 50.9µs ± 1% 19.2µs ± 6% -62.31% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.ParseReader-8 15.7µs ± 1% 6.0µs ± 4% -61.50% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.Parse-8 21.1µs ± 1% 8.3µs ± 2% -60.65% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.ParseString-8 21.3µs ± 2% 8.5µs ± 2% -59.84% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.ParseReader-8 21.5µs ± 1% 8.8µs ± 1% -59.03% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.ParseString-8 12.6µs ± 2% 5.2µs ± 2% -58.97% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.Parse-8 15.0µs ± 1% 6.2µs ± 2% -58.60% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.Parse-8 12.5µs ± 1% 5.2µs ± 2% -58.45% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.ParseReader-8 12.7µs ± 1% 5.4µs ± 5% -57.70% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.ParseString-8 15.1µs ± 1% 6.7µs ± 5% -55.91% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.ParseReader-8 15.0µs ± 0% 6.7µs ± 2% -55.32% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.ParseString-8 8.63µs ± 1% 4.17µs ± 2% -51.69% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.ParseString-8 8.52µs ± 1% 4.14µs ± 2% -51.47% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.ParseReader-8 8.63µs ± 2% 4.23µs ± 1% -51.01% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.Parse-8 8.32µs ± 1% 4.10µs ± 1% -50.70% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.Parse-8 8.49µs ± 0% 4.22µs ± 3% -50.34% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.ParseReader-8 8.46µs ± 1% 4.28µs ± 2% -49.36% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.Parse-8 4.23µs ± 1% 2.56µs ± 1% -39.41% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.Parse-8 8.13µs ± 1% 4.94µs ± 0% -39.23% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.ParseString-8 9.69µs ± 1% 5.97µs ± 1% -38.40% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.ParseString-8 8.24µs ± 1% 5.09µs ± 0% -38.25% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.Parse-8 9.68µs ± 1% 5.99µs ± 2% -38.14% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.ParseString-8 4.32µs ± 1% 2.67µs ± 2% -38.06% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.ParseReader-8 8.15µs ± 1% 5.08µs ± 1% -37.73% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.ParseReader-8 4.31µs ± 1% 2.70µs ± 1% -37.37% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.ParseReader-8 9.87µs ± 1% 6.24µs ± 2% -36.80% (p=0.008 n=5+5)
JWS/Serialization/JSON/json.Marshal-8 11.5µs ± 2% 11.2µs ± 1% -2.42% (p=0.016 n=5+5)
JWT/Serialization/JSON/json.Marshal-8 3.41µs ± 0% 3.38µs ± 1% -0.90% (p=0.016 n=5+5)
JWE/Serialization/JSON/json.Marshal-8 11.9µs ± 2% 11.8µs ± 6% ~ (p=1.000 n=5+5)
JWK/Serialization/EC/PublicKey/json.Marshal-8 5.43µs ± 1% 5.44µs ± 0% ~ (p=0.730 n=5+4)
JWK/Serialization/Symmetric/PublicKey/json.Marshal-8 4.16µs ± 1% 4.09µs ± 2% ~ (p=0.056 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/json.Marshal-8 4.15µs ± 1% 4.14µs ± 1% ~ (p=0.746 n=5+5)
JWT/Serialization/Sign/jwt.Sign-8 988µs ± 3% 980µs ± 6% ~ (p=0.310 n=5+5)
JWT/Serialization/JSON/json.Unmarshal-8 1.26µs ± 0% 1.30µs ± 1% +3.00% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/json.Marshal-8 6.28µs ± 2% 6.62µs ± 5% +5.35% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/json.Marshal-8 15.4µs ± 1% 16.7µs ±13% +8.86% (p=0.008 n=5+5)
JWE/Serialization/JSON/json.Unmarshal-8 4.57µs ± 1% 5.08µs ± 5% +11.11% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/json.Marshal-8 5.90µs ± 2% 7.03µs ± 3% +19.17% (p=0.008 n=5+5)
name old alloc/op new alloc/op delta
JWK/Serialization/RSA/PrivateKey/jwk.Parse-8 36.5kB ± 0% 23.9kB ± 0% -34.40% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/jwk.ParseString-8 38.3kB ± 0% 25.7kB ± 0% -32.79% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/jwk.ParseReader-8 41.1kB ± 0% 28.5kB ± 0% -30.54% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.Parse-8 10.5kB ± 0% 8.3kB ± 0% -20.72% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.ParseString-8 11.0kB ± 0% 8.8kB ± 0% -19.87% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.ParseReader-8 11.0kB ± 0% 8.8kB ± 0% -19.75% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.Parse-8 10.1kB ± 0% 8.3kB ± 0% -17.76% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.ParseString-8 10.1kB ± 0% 8.3kB ± 0% -17.68% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.ParseReader-8 10.6kB ± 0% 8.8kB ± 0% -16.91% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.Parse-8 16.3kB ± 0% 15.0kB ± 0% -8.22% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.ParseString-8 17.2kB ± 0% 15.9kB ± 0% -7.79% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.ParseReader-8 17.9kB ± 0% 16.5kB ± 0% -7.51% (p=0.000 n=5+4)
JWK/Serialization/RSA/PrivateKey/json.Marshal-8 8.83kB ± 0% 8.82kB ± 0% -0.13% (p=0.016 n=4+5)
JWE/Serialization/JSON/json.Marshal-8 3.96kB ± 0% 3.96kB ± 0% -0.11% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.Sign-8 36.2kB ± 0% 36.2kB ± 0% -0.08% (p=0.008 n=5+5)
JWS/Serialization/JSON/json.Marshal-8 5.17kB ± 0% 5.17kB ± 0% -0.08% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/json.Marshal-8 1.79kB ± 0% 1.79kB ± 0% -0.06% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/json.Marshal-8 2.28kB ± 0% 2.28kB ± 0% -0.04% (p=0.000 n=5+4)
JWK/Serialization/RSA/PublicKey/json.Marshal-8 2.34kB ± 0% 2.34kB ± 0% -0.04% (p=0.008 n=5+5)
JWE/Serialization/JSON/json.Unmarshal-8 1.58kB ± 0% 1.58kB ± 0% ~ (all equal)
JWK/Serialization/Symmetric/PublicKey/json.Marshal-8 1.15kB ± 0% 1.15kB ± 0% ~ (all equal)
JWK/Serialization/Symmetric/PrivateKey/json.Marshal-8 1.15kB ± 0% 1.15kB ± 0% ~ (p=0.444 n=5+5)
JWT/Serialization/JSON/json.Unmarshal-8 592B ± 0% 592B ± 0% ~ (all equal)
JWT/Serialization/JSON/json.Marshal-8 720B ± 0% 720B ± 0% ~ (all equal)
JWS/Serialization/Compact/jws.ParseReader-8 3.34kB ± 0% 3.49kB ± 0% +4.31% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.ParseString-8 3.02kB ± 0% 3.17kB ± 0% +4.76% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.Parse-8 2.83kB ± 0% 2.98kB ± 0% +5.08% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.ParseReader-8 8.13kB ± 0% 8.58kB ± 0% +5.51% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.ParseString-8 7.97kB ± 0% 8.42kB ± 0% +5.62% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.Parse-8 7.62kB ± 0% 8.06kB ± 0% +5.88% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.ParseReader-8 7.52kB ± 0% 8.03kB ± 0% +6.81% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.ParseString-8 7.25kB ± 0% 7.76kB ± 0% +7.06% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.Parse-8 7.01kB ± 0% 7.52kB ± 0% +7.31% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.ParseReader-8 6.75kB ± 0% 7.52kB ± 0% +11.37% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.ParseReader-8 6.75kB ± 0% 7.52kB ± 0% +11.37% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.ParseReader-8 7.70kB ± 0% 8.59kB ± 0% +11.64% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.ParseString-8 7.60kB ± 0% 8.50kB ± 0% +11.79% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.ParseString-8 6.37kB ± 0% 7.14kB ± 0% +12.06% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.ParseString-8 6.37kB ± 0% 7.14kB ± 0% +12.06% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.Parse-8 6.24kB ± 0% 7.01kB ± 0% +12.31% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.Parse-8 6.24kB ± 0% 7.01kB ± 0% +12.31% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.Parse-8 7.18kB ± 0% 8.08kB ± 0% +12.47% (p=0.008 n=5+5)
name old allocs/op new allocs/op delta
JWK/Serialization/RSA/PrivateKey/jwk.Parse-8 205 ± 0% 83 ± 0% -59.51% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/jwk.ParseString-8 206 ± 0% 84 ± 0% -59.22% (p=0.008 n=5+5)
JWK/Serialization/RSA/PrivateKey/jwk.ParseReader-8 209 ± 0% 87 ± 0% -58.37% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.Parse-8 129 ± 0% 59 ± 0% -54.26% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.ParseString-8 130 ± 0% 60 ± 0% -53.85% (p=0.008 n=5+5)
JWK/Serialization/EC/PrivateKey/jwk.ParseReader-8 130 ± 0% 60 ± 0% -53.85% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.Parse-8 112 ± 0% 54 ± 0% -51.79% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.ParseString-8 113 ± 0% 55 ± 0% -51.33% (p=0.008 n=5+5)
JWK/Serialization/EC/PublicKey/jwk.ParseReader-8 113 ± 0% 55 ± 0% -51.33% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.Parse-8 96.0 ± 0% 50.0 ± 0% -47.92% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.ParseString-8 97.0 ± 0% 51.0 ± 0% -47.42% (p=0.008 n=5+5)
JWK/Serialization/RSA/PublicKey/jwk.ParseReader-8 97.0 ± 0% 51.0 ± 0% -47.42% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.Parse-8 49.0 ± 0% 27.0 ± 0% -44.90% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.ParseString-8 50.0 ± 0% 28.0 ± 0% -44.00% (p=0.008 n=5+5)
JWS/Serialization/Compact/jws.ParseReader-8 50.0 ± 0% 28.0 ± 0% -44.00% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.Parse-8 81.0 ± 0% 47.0 ± 0% -41.98% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.Parse-8 81.0 ± 0% 47.0 ± 0% -41.98% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.ParseString-8 82.0 ± 0% 48.0 ± 0% -41.46% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PublicKey/jwk.ParseReader-8 82.0 ± 0% 48.0 ± 0% -41.46% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.ParseString-8 82.0 ± 0% 48.0 ± 0% -41.46% (p=0.008 n=5+5)
JWK/Serialization/Symmetric/PrivateKey/jwk.ParseReader-8 82.0 ± 0% 48.0 ± 0% -41.46% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.Parse-8 153 ± 0% 96 ± 0% -37.25% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.ParseString-8 154 ± 0% 97 ± 0% -37.01% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.Parse-8 103 ± 0% 65 ± 0% -36.89% (p=0.008 n=5+5)
JWS/Serialization/JSON/jws.ParseReader-8 155 ± 0% 98 ± 0% -36.77% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.ParseString-8 104 ± 0% 66 ± 0% -36.54% (p=0.008 n=5+5)
JWT/Serialization/Sign/jwt.ParseReader-8 104 ± 0% 66 ± 0% -36.54% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.Parse-8 73.0 ± 0% 58.0 ± 0% -20.55% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.ParseString-8 74.0 ± 0% 59.0 ± 0% -20.27% (p=0.008 n=5+5)
JWT/Serialization/JSON/jwt.ParseReader-8 74.0 ± 0% 59.0 ± 0% -20.27% (p=0.008 n=5+5)
JWE/Serialization/JSON/json.Marshal-8 45.0 ± 0% 45.0 ± 0% ~ (all equal)
JWE/Serialization/JSON/json.Unmarshal-8 26.0 ± 0% 26.0 ± 0% ~ (all equal)
JWK/Serialization/RSA/PublicKey/json.Marshal-8 24.0 ± 0% 24.0 ± 0% ~ (all equal)
JWK/Serialization/RSA/PrivateKey/json.Marshal-8 51.0 ± 0% 51.0 ± 0% ~ (all equal)
JWK/Serialization/EC/PublicKey/json.Marshal-8 27.0 ± 0% 27.0 ± 0% ~ (all equal)
JWK/Serialization/EC/PrivateKey/json.Marshal-8 31.0 ± 0% 31.0 ± 0% ~ (all equal)
JWK/Serialization/Symmetric/PublicKey/json.Marshal-8 20.0 ± 0% 20.0 ± 0% ~ (all equal)
JWK/Serialization/Symmetric/PrivateKey/json.Marshal-8 20.0 ± 0% 20.0 ± 0% ~ (all equal)
JWS/Serialization/JSON/json.Marshal-8 60.0 ± 0% 60.0 ± 0% ~ (all equal)
JWT/Serialization/Sign/jwt.Sign-8 192 ± 0% 192 ± 0% ~ (all equal)
JWT/Serialization/JSON/json.Unmarshal-8 10.0 ± 0% 10.0 ± 0% ~ (all equal)
JWT/Serialization/JSON/json.Marshal-8 18.0 ± 0% 18.0 ± 0% ~ (all equal)
@goccy
Copy link

goccy commented Feb 10, 2021

JWK/Serialization/RSA/PublicKey/json.Marshal のベンチを手元でとってみたのですが、
プロファイル結果に json の文字が現れないくらいには JSONライブラリが関係なさそうな雰囲気を感じました。

他にも JWK/Serialization/RSA/PublicKey/jwk.Parse を手元で走らせると

$ go test  -tags jwx_goccy -bench JWK/Serialization/RSA/PublicKey/jwk.Parse$
goos: darwin
goarch: amd64
pkg: github.com/lestrrat-go/jwx/bench
BenchmarkJWK/Serialization/RSA/PublicKey/jwk.Parse-16             101191             11621 ns/op
PASS
ok      github.com/lestrrat-go/jwx/bench        7.830s
$ go test  -bench JWK/Serialization/RSA/PublicKey/jwk.Parse$
goos: darwin
goarch: amd64
pkg: github.com/lestrrat-go/jwx/bench
BenchmarkJWK/Serialization/RSA/PublicKey/jwk.Parse-16              59337             19931 ns/op
PASS
ok      github.com/lestrrat-go/jwx/bench        5.006s

みたいになって自分のライブラリの方が速かったりしたので、誤差の範囲だったりするかもですね。

@lestrrat
Copy link
Author

ふむふむ。まぁTwitterでも書きましたけど、+200%とかが消えたからそのあたりは確かに誤差なのかもしれませんね。
まぁ、一応あたまにとめておく、くらいな感じですかね

@goccy
Copy link

goccy commented Feb 10, 2021

そうですね。引き続き何か最適化する際に参考にさせていただきます！

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment