serdebench: PGO optimized compared to Release

gistfile1.txt

     Running benches/bench.rs (target/aarch64-apple-darwin/release/deps/bench-daa0c585b9c93960)
Gnuplot not found, using plotters backend
Benchmarking all/sr.json
Benchmarking all/sr.json: Warming up for 3.0000 s
Benchmarking all/sr.json: Collecting 100 samples in estimated 5.0003 s (69M iterations)
Benchmarking all/sr.json: Analyzing
all/sr.json             time:   [71.988 ns 72.314 ns 72.712 ns]
                        change: [-28.880% -28.598% -28.310%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe
json: 100 bytes
Benchmarking all/de.json
Benchmarking all/de.json: Warming up for 3.0000 s
Benchmarking all/de.json: Collecting 100 samples in estimated 5.0005 s (34M iterations)
Benchmarking all/de.json: Analyzing
all/de.json             time:   [146.71 ns 147.35 ns 148.17 ns]
                        change: [-26.550% -26.299% -25.993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
Benchmarking all/sr.simd-json
Benchmarking all/sr.simd-json: Warming up for 3.0000 s
Benchmarking all/sr.simd-json: Collecting 100 samples in estimated 5.0002 s (146M iterations)
Benchmarking all/sr.simd-json: Analyzing
all/sr.simd-json        time:   [34.298 ns 34.364 ns 34.447 ns]
                        change: [+17.925% +18.178% +18.500%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  3 (3.00%) high mild
  7 (7.00%) high severe
simd-json: 100 bytes
Benchmarking all/de.simd-json
Benchmarking all/de.simd-json: Warming up for 3.0000 s
Benchmarking all/de.simd-json: Collecting 100 samples in estimated 5.0000 s (14M iterations)
Benchmarking all/de.simd-json: Analyzing
all/de.simd-json        time:   [346.36 ns 347.73 ns 349.14 ns]
                        change: [-6.9846% -6.5120% -6.1029%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  6 (6.00%) high mild
  3 (3.00%) high severe
simd-json: 100 bytes
Benchmarking all/sr.yaml
Benchmarking all/sr.yaml: Warming up for 3.0000 s
Benchmarking all/sr.yaml: Collecting 100 samples in estimated 5.0012 s (4.0M iterations)
Benchmarking all/sr.yaml: Analyzing
all/sr.yaml             time:   [1.2452 us 1.2491 us 1.2538 us]
                        change: [-0.4297% +0.0619% +0.5181%] (p = 0.80 > 0.05)
                        No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) high mild
  6 (6.00%) high severe
yaml: 99 bytes
Benchmarking all/de.yaml
Benchmarking all/de.yaml: Warming up for 3.0000 s
Benchmarking all/de.yaml: Collecting 100 samples in estimated 5.0157 s (1.4M iterations)
Benchmarking all/de.yaml: Analyzing
all/de.yaml             time:   [3.3359 us 3.3509 us 3.3732 us]
                        change: [-7.8245% -7.5061% -7.2005%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking all/sr.ron
Benchmarking all/sr.ron: Warming up for 3.0000 s
Benchmarking all/sr.ron: Collecting 100 samples in estimated 5.0001 s (26M iterations)
Benchmarking all/sr.ron: Analyzing
all/sr.ron              time:   [189.54 ns 190.01 ns 190.49 ns]
                        change: [-19.994% -19.695% -19.411%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
ron: 91 bytes
Benchmarking all/de.ron
Benchmarking all/de.ron: Warming up for 3.0000 s
Benchmarking all/de.ron: Collecting 100 samples in estimated 5.0005 s (15M iterations)
Benchmarking all/de.ron: Analyzing
all/de.ron              time:   [335.66 ns 336.10 ns 336.61 ns]
                        change: [-20.784% -20.454% -20.173%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) high mild
  3 (3.00%) high severe
Benchmarking all/sr.bincode
Benchmarking all/sr.bincode: Warming up for 3.0000 s
Benchmarking all/sr.bincode: Collecting 100 samples in estimated 5.0000 s (424M iterations)
Benchmarking all/sr.bincode: Analyzing
all/sr.bincode          time:   [12.133 ns 12.280 ns 12.413 ns]
                        change: [+15.556% +16.971% +18.479%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  5 (5.00%) low mild
bincode: 58 bytes
Benchmarking all/de.bincode
Benchmarking all/de.bincode: Warming up for 3.0000 s
Benchmarking all/de.bincode: Collecting 100 samples in estimated 5.0001 s (115M iterations)
Benchmarking all/de.bincode: Analyzing
all/de.bincode          time:   [43.080 ns 43.309 ns 43.618 ns]
                        change: [-20.188% -19.758% -19.318%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) high mild
  1 (1.00%) high severe
Benchmarking all/sr.msgpack
Benchmarking all/sr.msgpack: Warming up for 3.0000 s
Benchmarking all/sr.msgpack: Collecting 100 samples in estimated 5.0000 s (357M iterations)
Benchmarking all/sr.msgpack: Analyzing
all/sr.msgpack          time:   [13.866 ns 13.891 ns 13.920 ns]
                        change: [-49.123% -48.965% -48.824%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
msgpack: 24 bytes
Benchmarking all/de.msgpack
Benchmarking all/de.msgpack: Warming up for 3.0000 s
Benchmarking all/de.msgpack: Collecting 100 samples in estimated 5.0001 s (73M iterations)
Benchmarking all/de.msgpack: Analyzing
all/de.msgpack          time:   [67.927 ns 68.048 ns 68.187 ns]
                        change: [-23.636% -23.374% -23.103%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
  8 (8.00%) high mild
  2 (2.00%) high severe
Benchmarking all/sr.cbor
Benchmarking all/sr.cbor: Warming up for 3.0000 s
Benchmarking all/sr.cbor: Collecting 100 samples in estimated 5.0001 s (282M iterations)
Benchmarking all/sr.cbor: Analyzing
all/sr.cbor             time:   [17.263 ns 17.299 ns 17.350 ns]
                        change: [-52.352% -52.228% -52.100%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) high mild
  6 (6.00%) high severe
cbor: 26 bytes
Benchmarking all/de.cbor
Benchmarking all/de.cbor: Warming up for 3.0000 s
Benchmarking all/de.cbor: Collecting 100 samples in estimated 5.0001 s (71M iterations)
Benchmarking all/de.cbor: Analyzing
all/de.cbor             time:   [70.257 ns 70.461 ns 70.656 ns]
                        change: [-13.297% -12.373% -11.233%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
Benchmarking all/sr.postcard
Benchmarking all/sr.postcard: Warming up for 3.0000 s
Benchmarking all/sr.postcard: Collecting 100 samples in estimated 5.0001 s (234M iterations)
Benchmarking all/sr.postcard: Analyzing
all/sr.postcard         time:   [21.274 ns 21.383 ns 21.494 ns]
                        change: [-29.260% -28.547% -27.933%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  3 (3.00%) high mild
postcard: 41 bytes
Benchmarking all/de.postcard
Benchmarking all/de.postcard: Warming up for 3.0000 s
Benchmarking all/de.postcard: Collecting 100 samples in estimated 5.0000 s (69M iterations)
Benchmarking all/de.postcard: Analyzing
all/de.postcard         time:   [70.934 ns 71.101 ns 71.269 ns]
                        change: [+16.648% +17.049% +17.486%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  4 (4.00%) high mild
  4 (4.00%) high severe
Benchmarking all/ser.flexbuffers
Benchmarking all/ser.flexbuffers: Warming up for 3.0000 s
Benchmarking all/ser.flexbuffers: Collecting 100 samples in estimated 5.0018 s (9.0M iterations)
Benchmarking all/ser.flexbuffers: Analyzing
all/ser.flexbuffers     time:   [551.76 ns 552.79 ns 554.06 ns]
                        change: [-19.548% -19.002% -18.518%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  6 (6.00%) high severe
flexbuffers: 41 bytes
Benchmarking all/de.flexbuffers
Benchmarking all/de.flexbuffers: Warming up for 3.0000 s
Benchmarking all/de.flexbuffers: Collecting 100 samples in estimated 5.0003 s (28M iterations)
Benchmarking all/de.flexbuffers: Analyzing
all/de.flexbuffers      time:   [177.79 ns 178.15 ns 178.57 ns]
                        change: [-34.922% -34.674% -34.453%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe
Benchmarking all/sr.flatbuffers
Benchmarking all/sr.flatbuffers: Warming up for 3.0000 s
Benchmarking all/sr.flatbuffers: Collecting 100 samples in estimated 5.0002 s (101M iterations)
Benchmarking all/sr.flatbuffers: Analyzing
all/sr.flatbuffers      time:   [49.534 ns 49.747 ns 49.980 ns]
                        change: [-23.541% -22.754% -22.117%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
  1 (1.00%) high severe
flatbuffers: 104 bytes
Benchmarking all/de.flatbuffers
Benchmarking all/de.flatbuffers: Warming up for 3.0000 s
Benchmarking all/de.flatbuffers: Collecting 100 samples in estimated 5.0002 s (100M iterations)
Benchmarking all/de.flatbuffers: Analyzing
all/de.flatbuffers      time:   [49.958 ns 50.140 ns 50.357 ns]
                        change: [-5.6552% -5.2918% -4.9354%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  4 (4.00%) high mild
  3 (3.00%) high severe
Benchmarking all/sr.capnproto.unpacked
Benchmarking all/sr.capnproto.unpacked: Warming up for 3.0000 s
Benchmarking all/sr.capnproto.unpacked: Collecting 100 samples in estimated 5.0003 s (90M iterations)
Benchmarking all/sr.capnproto.unpacked: Analyzing
all/sr.capnproto.unpacked
                        time:   [55.337 ns 55.429 ns 55.563 ns]
                        change: [-15.502% -15.293% -15.077%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe
capnproto.unpacked: 96 bytes
Benchmarking all/de.capnproto.unpacked
Benchmarking all/de.capnproto.unpacked: Warming up for 3.0000 s
Benchmarking all/de.capnproto.unpacked: Collecting 100 samples in estimated 5.0004 s (53M iterations)
Benchmarking all/de.capnproto.unpacked: Analyzing
all/de.capnproto.unpacked
                        time:   [93.253 ns 93.405 ns 93.561 ns]
                        change: [-36.003% -35.851% -35.705%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
Benchmarking all/sr.capnproto.packed
Benchmarking all/sr.capnproto.packed: Warming up for 3.0000 s
Benchmarking all/sr.capnproto.packed: Collecting 100 samples in estimated 5.0004 s (46M iterations)
Benchmarking all/sr.capnproto.packed: Analyzing
all/sr.capnproto.packed time:   [107.29 ns 107.45 ns 107.62 ns]
                        change: [-10.056% -9.7302% -9.4397%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  3 (3.00%) high severe
capnproto.packed: 39 bytes
Benchmarking all/de.capnproto.packed
Benchmarking all/de.capnproto.packed: Warming up for 3.0000 s
Benchmarking all/de.capnproto.packed: Collecting 100 samples in estimated 5.0006 s (24M iterations)
Benchmarking all/de.capnproto.packed: Analyzing
all/de.capnproto.packed time:   [203.66 ns 204.49 ns 205.58 ns]
                        change: [-18.413% -18.003% -17.563%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
  5 (5.00%) high mild
  3 (3.00%) high severe
Benchmarking all/sr.proto3
Benchmarking all/sr.proto3: Warming up for 3.0000 s
Benchmarking all/sr.proto3: Collecting 100 samples in estimated 5.0001 s (94M iterations)
Benchmarking all/sr.proto3: Analyzing
all/sr.proto3           time:   [52.889 ns 53.044 ns 53.241 ns]
                        change: [-22.485% -22.254% -22.003%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  4 (4.00%) high mild
  9 (9.00%) high severe
proto3: 23 bytes
Benchmarking all/de.proto3
Benchmarking all/de.proto3: Warming up for 3.0000 s
Benchmarking all/de.proto3: Collecting 100 samples in estimated 5.0002 s (54M iterations)
Benchmarking all/de.proto3: Analyzing
all/de.proto3           time:   [92.026 ns 92.380 ns 92.754 ns]
                        change: [-9.3924% -8.8449% -8.3763%] (p = 0.00 < 0.05)
                        Performance has improved.
Benchmarking all/sr.abomonation
Benchmarking all/sr.abomonation: Warming up for 3.0000 s
Benchmarking all/sr.abomonation: Collecting 100 samples in estimated 5.0000 s (544M iterations)
Benchmarking all/sr.abomonation: Analyzing
all/sr.abomonation      time:   [9.1874 ns 9.2213 ns 9.2576 ns]
                        change: [-4.1753% -3.6324% -3.1145%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
abomonation: 108 bytes
Benchmarking all/de.abomonation
Benchmarking all/de.abomonation: Warming up for 3.0000 s
Benchmarking all/de.abomonation: Collecting 100 samples in estimated 5.0000 s (1.1B iterations)
Benchmarking all/de.abomonation: Analyzing
all/de.abomonation      time:   [4.5044 ns 4.5225 ns 4.5431 ns]
                        change: [+10.345% +10.967% +11.597%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking all/sr.rkyv
Benchmarking all/sr.rkyv: Warming up for 3.0000 s
Benchmarking all/sr.rkyv: Collecting 100 samples in estimated 5.0000 s (430M iterations)
Benchmarking all/sr.rkyv: Analyzing
all/sr.rkyv             time:   [11.551 ns 11.574 ns 11.602 ns]
                        change: [-35.274% -35.048% -34.808%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) high mild
  5 (5.00%) high severe
rkyv: 72 bytes
Benchmarking all/de.rkyv (unvalidated)
Benchmarking all/de.rkyv (unvalidated): Warming up for 3.0000 s
Benchmarking all/de.rkyv (unvalidated): Collecting 100 samples in estimated 5.0000 s (8.3B iterations)
Benchmarking all/de.rkyv (unvalidated): Analyzing
all/de.rkyv (unvalidated)
                        time:   [599.81 ps 600.66 ps 601.84 ps]
                        change: [-1.0014% -0.7213% -0.4366%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) high mild
  4 (4.00%) high severe
Benchmarking all/de.rkyv (validated)
Benchmarking all/de.rkyv (validated): Warming up for 3.0000 s
Benchmarking all/de.rkyv (validated): Collecting 100 samples in estimated 5.0000 s (304M iterations)
Benchmarking all/de.rkyv (validated): Analyzing
all/de.rkyv (validated) time:   [16.317 ns 16.339 ns 16.367 ns]
                        change: [-10.794% -10.349% -9.9021%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
  2 (2.00%) high mild
  7 (7.00%) high severe