Supranational BLST Benchmarks (portable)

portable_bls_benchmarks.txt

     Running target/release/deps/blst_benches-a4084c38c76b1d19
Gnuplot not found, using plotters backend
verify_multi_aggregate/verify_multi_aggregate/8                                                                             
                        time:   [3.4410 ms 3.4688 ms 3.5025 ms]
                        change: [+4.5451% +5.4146% +6.4445%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe
verify_multi_aggregate/verify_multi_aggregate/16                                                                            
                        time:   [6.7284 ms 6.7917 ms 6.8582 ms]
                        change: [+19.218% +21.411% +23.495%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 23 outliers among 100 measurements (23.00%)
  10 (10.00%) low severe
  1 (1.00%) high mild
  12 (12.00%) high severe
verify_multi_aggregate/verify_multi_aggregate/32                                                                            
                        time:   [12.087 ms 12.146 ms 12.208 ms]
                        change: [+26.646% +27.383% +28.099%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
verify_multi_aggregate/verify_multi_aggregate/64                                                                            
                        time:   [22.645 ms 22.689 ms 22.737 ms]
                        change: [+26.345% +26.948% +27.570%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  3 (3.00%) high mild
  5 (5.00%) high severe
verify_multi_aggregate/verify_multi_aggregate/128                                                                            
                        time:   [44.201 ms 44.342 ms 44.488 ms]
                        change: [+27.171% +27.767% +28.369%] (p = 0.00 < 0.05)
                        Performance has regressed.

Benchmarking fast_aggregate_verify/fast_aggregate_verify/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.1s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify/8                                                                             
                        time:   [1.4061 ms 1.4155 ms 1.4245 ms]
                        change: [+23.561% +24.471% +25.330%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking fast_aggregate_verify/fast_aggregate_verify_preagg/8: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.0s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify_preagg/8                                                                             
                        time:   [1.4239 ms 1.4411 ms 1.4573 ms]
                        change: [+5.1381% +8.4157% +11.700%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  2 (2.00%) low mild
  1 (1.00%) high severe
Benchmarking fast_aggregate_verify/fast_aggregate_verify/16: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.4s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify/16                                                                             
                        time:   [1.4524 ms 1.4623 ms 1.4723 ms]
                        change: [+21.932% +24.493% +26.957%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking fast_aggregate_verify/fast_aggregate_verify_preagg/16: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.1s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify_preagg/16                                                                             
                        time:   [1.4026 ms 1.4148 ms 1.4257 ms]
                        change: [+21.181% +22.546% +23.927%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking fast_aggregate_verify/fast_aggregate_verify/32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.4s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify/32                                                                             
                        time:   [1.4280 ms 1.4468 ms 1.4653 ms]
                        change: [+18.951% +20.336% +21.704%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  7 (7.00%) high mild
Benchmarking fast_aggregate_verify/fast_aggregate_verify_preagg/32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.2s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify_preagg/32                                                                             
                        time:   [1.4180 ms 1.4350 ms 1.4505 ms]
                        change: [+27.698% +28.974% +30.344%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
  1 (1.00%) high mild
  2 (2.00%) high severe
Benchmarking fast_aggregate_verify/fast_aggregate_verify/64: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.6s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify/64                                                                             
                        time:   [1.4902 ms 1.5010 ms 1.5104 ms]
                        change: [+23.063% +24.541% +25.964%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking fast_aggregate_verify/fast_aggregate_verify_preagg/64: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.3s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify_preagg/64                                                                             
                        time:   [1.4361 ms 1.4473 ms 1.4583 ms]
                        change: [+29.066% +30.578% +32.026%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low mild
  1 (1.00%) high mild
Benchmarking fast_aggregate_verify/fast_aggregate_verify/128: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 8.0s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify/128                                                                             
                        time:   [1.5431 ms 1.5574 ms 1.5719 ms]
                        change: [+26.144% +27.504% +28.905%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
Benchmarking fast_aggregate_verify/fast_aggregate_verify_preagg/128: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.4s, enable flat sampling, or reduce sample count to 50.
fast_aggregate_verify/fast_aggregate_verify_preagg/128                                                                             
                        time:   [1.3761 ms 1.3912 ms 1.4072 ms]
                        change: [+23.523% +25.215% +26.831%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild

aggregate_verify/aggregate_verify/8                                                                             
                        time:   [3.1603 ms 3.1917 ms 3.2235 ms]
                        change: [+21.105% +22.882% +24.792%] (p = 0.00 < 0.05)
                        Performance has regressed.
aggregate_verify/aggregate_verify/16                                                                            
                        time:   [4.5730 ms 4.6259 ms 4.6848 ms]
                        change: [+9.9970% +11.580% +13.252%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
  3 (3.00%) low mild
  6 (6.00%) high severe
aggregate_verify/aggregate_verify/32                                                                            
                        time:   [8.8056 ms 8.8415 ms 8.8780 ms]
                        change: [+25.097% +25.941% +26.811%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
aggregate_verify/aggregate_verify/64                                                                            
                        time:   [16.220 ms 16.249 ms 16.283 ms]
                        change: [+26.959% +27.575% +28.201%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  6 (6.00%) high severe
aggregate_verify/aggregate_verify/128                                                                            
                        time:   [31.687 ms 31.951 ms 32.242 ms]
                        change: [+28.605% +29.769% +31.085%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
  7 (7.00%) high mild
  3 (3.00%) high severe

aggregate/aggregate_signature/10                                                                             
                        time:   [18.921 us 18.937 us 18.957 us]
                        change: [+24.661% +25.719% +26.490%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  5 (5.00%) high mild
  2 (2.00%) high severe
aggregate/aggregate_public_key/10                                                                             
                        time:   [7.2939 us 7.3021 us 7.3121 us]
                        change: [+42.882% +43.641% +44.253%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  1 (1.00%) low severe
  2 (2.00%) high mild
  1 (1.00%) high severe
aggregate/aggregate_signature/50                                                                            
                        time:   [101.59 us 101.72 us 101.88 us]
                        change: [+24.093% +24.346% +24.628%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high severe
aggregate/aggregate_public_key/50                                                                             
                        time:   [38.871 us 38.916 us 38.963 us]
                        change: [+43.083% +43.305% +43.547%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  3 (3.00%) high mild
  1 (1.00%) high severe
aggregate/aggregate_signature/100                                                                            
                        time:   [213.09 us 213.35 us 213.65 us]
                        change: [+32.753% +34.761% +37.564%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  4 (4.00%) high mild
  8 (8.00%) high severe
aggregate/aggregate_public_key/100                                                                            
                        time:   [78.168 us 78.249 us 78.350 us]
                        change: [+35.152% +35.618% +36.071%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
aggregate/aggregate_signature/300                                                                            
                        time:   [647.69 us 648.39 us 649.20 us]
                        change: [+33.682% +34.027% +34.361%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  6 (6.00%) high mild
  1 (1.00%) high severe
aggregate/aggregate_public_key/300                                                                            
                        time:   [234.31 us 234.64 us 235.00 us]
                        change: [+34.080% +34.683% +35.255%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
aggregate/aggregate_signature/1000                                                                             
                        time:   [2.1517 ms 2.1712 ms 2.2074 ms]
                        change: [+29.563% +31.026% +33.391%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  4 (4.00%) high severe
aggregate/aggregate_public_key/1000                                                                            
                        time:   [786.20 us 787.59 us 789.18 us]
                        change: [+36.301% +36.593% +36.859%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
aggregate/aggregate_signature/4000                                                                            
                        time:   [8.6723 ms 8.6879 ms 8.7091 ms]
                        change: [+30.967% +31.721% +32.354%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
aggregate/aggregate_public_key/4000                                                                             
                        time:   [3.0145 ms 3.0182 ms 3.0219 ms]
                        change: [+29.880% +30.072% +30.254%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild

single_message/sign     time:   [860.85 us 861.75 us 862.81 us]                                
                        change: [+31.301% +32.601% +33.563%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe
Benchmarking single_message/verify: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 7.0s, enable flat sampling, or reduce sample count to 50.
single_message/verify   time:   [1.3759 ms 1.3873 ms 1.3999 ms]                                   
                        change: [+20.240% +21.595% +22.994%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) high mild
  1 (1.00%) high severe

serdes/secret_key_serialize                                                                             
                        time:   [38.610 ns 38.653 ns 38.698 ns]
                        change: [+15.123% +15.461% +15.889%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
  2 (2.00%) low mild
  2 (2.00%) high mild
  3 (3.00%) high severe
serdes/secret_key_deserialize                                                                             
                        time:   [35.620 ns 35.685 ns 35.754 ns]
                        change: [-0.2689% -0.0090% +0.2471%] (p = 0.95 > 0.05)
                        No change in performance detected.
Found 7 outliers among 100 measurements (7.00%)
  1 (1.00%) low mild
  4 (4.00%) high mild
  2 (2.00%) high severe
serdes/public_key_serialize                                                                            
                        time:   [235.37 ns 235.73 ns 236.11 ns]
                        change: [+23.684% +23.971% +24.267%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  2 (2.00%) high mild
serdes/public_key_compress                                                                            
                        time:   [140.01 ns 140.25 ns 140.51 ns]
                        change: [+26.457% +26.926% +27.345%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
  4 (4.00%) low severe
  6 (6.00%) low mild
  1 (1.00%) high mild
  2 (2.00%) high severe
serdes/public_key_uncompress                                                                             
                        time:   [21.083 us 21.105 us 21.129 us]
                        change: [+42.153% +42.550% +42.894%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
  1 (1.00%) low severe
  1 (1.00%) high severe
serdes/public_key_deserialize                                                                            
                        time:   [336.21 ns 336.74 ns 337.41 ns]
                        change: [+14.729% +15.505% +16.665%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
  9 (9.00%) high mild
  3 (3.00%) high severe
serdes/signature_serialize                                                                            
                        time:   [394.87 ns 395.35 ns 395.91 ns]
                        change: [+25.507% +25.774% +26.071%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  3 (3.00%) high mild
  2 (2.00%) high severe
serdes/signature_compress                                                                            
                        time:   [302.68 ns 304.14 ns 305.95 ns]
                        change: [+23.758% +24.408% +25.187%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  3 (3.00%) high mild
  11 (11.00%) high severe
serdes/signature_uncompress                                                                            
                        time:   [99.425 us 99.564 us 99.736 us]
                        change: [+34.421% +35.136% +35.646%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low severe
  3 (3.00%) high mild
  1 (1.00%) high severe
serdes/signature_deserialize                                                                             
                        time:   [793.39 ns 794.63 ns 796.37 ns]
                        change: [+23.507% +23.844% +24.164%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
serdes/p1_serialize     time:   [21.274 us 21.301 us 21.333 us]                                 
                        change: [+42.867% +43.101% +43.345%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  2 (2.00%) low severe
  2 (2.00%) high mild
  1 (1.00%) high severe
serdes/p1_compress      time:   [21.036 us 21.062 us 21.089 us]                                
                        change: [+43.747% +44.017% +44.291%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
  1 (1.00%) low mild
  3 (3.00%) high mild
  1 (1.00%) high severe
serdes/p2_serialize     time:   [22.136 us 22.154 us 22.173 us]                                 
                        change: [+41.518% +42.309% +42.903%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
  6 (6.00%) high mild
  8 (8.00%) high severe
serdes/p2_compress      time:   [20.812 us 20.837 us 20.866 us]                                
                        change: [+35.094% +35.331% +35.570%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
  2 (2.00%) high mild
  2 (2.00%) high severe

keys/key_gen            time:   [2.0918 us 2.0959 us 2.1004 us]                          
                        change: [+4.3054% +4.5413% +4.7664%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild
keys/sk_to_pk           time:   [200.50 us 200.82 us 201.17 us]                          
                        change: [+28.264% +28.799% +29.249%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
  5 (5.00%) high mild
  1 (1.00%) high severe
keys/key_validate       time:   [94.274 us 94.378 us 94.501 us]                              
                        change: [+27.634% +27.959% +28.248%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
  1 (1.00%) high mild