Skip to content

Instantly share code, notes, and snippets.

@zac-williamson
Last active June 21, 2023 10:44
Show Gist options
  • Save zac-williamson/838410a3da179d47d31b25b586c15e53 to your computer and use it in GitHub Desktop.
Save zac-williamson/838410a3da179d47d31b25b586c15e53 to your computer and use it in GitHub Desktop.
Result of parity benchmarks after `pairing_batch`, `precompute` and `miller_loop` optimizations

03 May 2019, Updated Parity Benchmarks

These Parity client benchmarks were run to evaluate the optimizations made to the Parity bn pairing library (link here)

By comparing pre- and post- benchmarks there is a significant improvement to the speed of multiple bilinear pairings that merits a reduction in the gas schedule.

The new benchmarks were run on branches that are currently outstanding pull requests into the Parity client (openethereum/parity-ethereum#10624) and the BN library (paritytech/bn#14).

The previous benchmarks were computed on 25 Oct 2018

2.1.1-beta (release 20 Sep)

  • master branch from 25 Oct (5a1dc3e)
  • bn pointing to 2a71dbd
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
ecrecover 3000.0 116516.00 18.8587916544 1165.16 3000.0
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
modexp_nagydani_1_qube 204.0 8188.00 24.91450904 81.88 210.8208315
alt_bn128_add_cdetrio12 500.0 5551.00 90.07386057 55.51 142.9245769
alt_bn128_mul_cdetrio11 40000.0 229342.00 174.4120135 2293.42 5904.991589
alt_bn128_pairing_ten_point_match_1 900000.0 34076695.00 26.41101198 340766.95 877390.9592

After pairing optimizations (05 May 2019)

  • parity-ethereum commit hash f301f365987fd214f2f43d9d5f1f6ad54fbe1c14 (on a fork of parity-ethereum)
  • bn pointing to commit hash 65d150fd3ee3694682060defbfc4dc74754690f0 (add-wnaf branch of a fork of bn)
  • Both forks have outstanding pull requests to the respective Parity repositories
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
ecrecover 3000.0 116640.00 25.72 1166.40 3000.00
Name Gascost Time (ns) MGas/S Gascost for 10MGas/S Gascost for ECDSA eq
modexp_nagydani_1_qube 204.0 40597.00 5.03 405.97 1044.16
alt_bn128_add_cdetrio12 500.0 6932.50 72.12 69.33 178.31
alt_bn128_mul_cdetrio11 40000.0 232660.00 171.92 2326.60 5984.05
alt_bn128_pairing_one_point 180000.0 3036800.00 59.27 30368.00 78107.00
alt_bn128_pairing_ten_point_match_1 900000.0 14663000.00 61.38 146630.00 377134.77

Computing pairing costs relative to ecrecover

  • ecrecover-equiv-cost of 1 pairing = 78,107
  • ecrecover-equiv-cost of 10 pairings = 377,134
  • set gas-cost = A + (B * number_pairings)
  • A + 10B = 377,134
  • A + B = 78,107
  • B = 33,225
  • A = 44,882
$ cargo bench --package ethcore
test result: ok. 0 passed; 0 failed; 315 ignored; 0 measured; 0 filtered out
Running target/release/deps/builtin-63274a173a57e4ec
ecrecover time: [115.64 us 116.64 us 117.84 us]
change: [-15.538% -11.932% -8.4109%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
sha256 time: [469.14 ns 470.14 ns 471.26 ns]
change: [-1.4282% -0.8673% -0.3296%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
4 (4.00%) high mild
6 (6.00%) high severe
ripemd time: [634.48 ns 636.32 ns 638.53 ns]
change: [-1.7531% -1.0084% -0.3099%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 13 outliers among 100 measurements (13.00%)
7 (7.00%) high mild
6 (6.00%) high severe
identity time: [95.023 ns 95.219 ns 95.434 ns]
change: [-0.5492% -0.1407% +0.3080%] (p = 0.54 > 0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
3 (3.00%) low mild
5 (5.00%) high mild
5 (5.00%) high severe
modexp_eip_example1 time: [342.59 us 347.65 us 353.42 us]
change: [+1.2207% +2.5815% +3.9728%] (p = 0.00 < 0.05)
Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
8 (8.00%) high mild
9 (9.00%) high severe
modexp_eip_example2 time: [209.38 ns 216.23 ns 224.52 ns]
change: [+18.815% +21.650% +24.322%] (p = 0.00 < 0.05)
Performance has regressed.
modexp_nagydani_1_square
time: [3.4935 us 3.5031 us 3.5152 us]
change: [-0.2591% +0.4189% +1.1853%] (p = 0.26 > 0.05)
No change in performance detected.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
modexp_nagydani_1_qube time: [6.7977 us 6.9930 us 7.1784 us]
change: [+6.4921% +8.7898% +11.323%] (p = 0.00 < 0.05)
Performance has regressed.
modexp_nagydani_1_pow0x10001
time: [40.466 us 40.597 us 40.766 us]
change: [-0.6311% -0.0183% +0.7407%] (p = 0.96 > 0.05)
No change in performance detected.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
modexp_nagydani_2_square
time: [6.7436 us 6.8670 us 6.9948 us]
change: [-1.3922% +0.4736% +2.2082%] (p = 0.62 > 0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
2 (2.00%) high mild
11 (11.00%) high severe
modexp_nagydani_2_qube time: [12.013 us 12.222 us 12.496 us]
change: [+14.653% +18.209% +21.687%] (p = 0.00 < 0.05)
Performance has regressed.
modexp_nagydani_2_pow0x10001
time: [88.797 us 89.031 us 89.306 us]
change: [-1.4859% -0.3746% +0.5837%] (p = 0.52 > 0.05)
No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) high mild
6 (6.00%) high severe
modexp_nagydani_3_square
time: [16.818 us 16.866 us 16.924 us]
change: [+1.9029% +2.6082% +3.4648%] (p = 0.00 < 0.05)
Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
9 (9.00%) high mild
5 (5.00%) high severe
modexp_nagydani_3_qube time: [31.633 us 31.769 us 31.939 us]
change: [+0.7613% +2.5481% +4.6156%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
2 (2.00%) high mild
10 (10.00%) high severe
modexp_nagydani_3_pow0x10001
time: [258.16 us 261.33 us 265.21 us]
change: [+8.6337% +10.646% +12.779%] (p = 0.00 < 0.05)
Performance has regressed.
modexp_nagydani_4_square
time: [48.384 us 48.510 us 48.657 us]
change: [-0.5903% +0.3551% +1.1182%] (p = 0.46 > 0.05)
No change in performance detected.
Found 13 outliers among 100 measurements (13.00%)
9 (9.00%) high mild
4 (4.00%) high severe
modexp_nagydani_4_qube time: [93.825 us 94.359 us 94.995 us]
change: [+1.0922% +1.7758% +2.4515%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
modexp_nagydani_4_pow0x10001
time: [761.67 us 764.26 us 767.40 us]
change: [-0.9126% -0.2773% +0.3024%] (p = 0.39 > 0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
5 (5.00%) high mild
6 (6.00%) high severe
modexp_nagydani_5_square
time: [162.64 us 163.43 us 164.49 us]
change: [-3.7298% -1.8336% -0.3598%] (p = 0.02 < 0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) high mild
5 (5.00%) high severe
modexp_nagydani_5_qube time: [320.70 us 331.24 us 350.50 us]
change: [+0.6416% +3.1053% +5.9900%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 16 outliers among 100 measurements (16.00%)
2 (2.00%) high mild
14 (14.00%) high severe
modexp_nagydani_5_pow0x10001
time: [2.6686 ms 2.6731 ms 2.6781 ms]
change: [-4.8550% -2.2522% -0.3098%] (p = 0.05 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
alt_bn128_add_chfast1 time: [5.4961 us 5.5111 us 5.5289 us]
change: [-1.5603% -0.7805% +0.0833%] (p = 0.06 > 0.05)
No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
3 (3.00%) high mild
9 (9.00%) high severe
alt_bn128_add_chfast2 time: [5.6512 us 5.6725 us 5.7023 us]
change: [-1.5864% -0.9880% -0.3701%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 10 outliers among 100 measurements (10.00%)
2 (2.00%) high mild
8 (8.00%) high severe
alt_bn128_add_cdetrio1 time: [159.34 ns 160.11 ns 161.10 ns]
change: [+2.7805% +4.3479% +6.0450%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
alt_bn128_add_cdetrio2 time: [165.06 ns 165.63 ns 166.32 ns]
change: [+1.5021% +2.1923% +2.8904%] (p = 0.00 < 0.05)
Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
5 (5.00%) high mild
7 (7.00%) high severe
alt_bn128_add_cdetrio3 time: [168.72 ns 169.31 ns 170.05 ns]
change: [+1.5674% +2.6017% +3.3940%] (p = 0.00 < 0.05)
Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
6 (6.00%) high mild
5 (5.00%) high severe
alt_bn128_add_cdetrio4 time: [169.06 ns 169.40 ns 169.76 ns]
change: [+2.4895% +3.1333% +3.8183%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
2 (2.00%) high mild
5 (5.00%) high severe
alt_bn128_add_cdetrio5 time: [162.43 ns 163.21 ns 164.34 ns]
change: [-0.2508% +0.4091% +1.2716%] (p = 0.31 > 0.05)
No change in performance detected.
Found 14 outliers among 100 measurements (14.00%)
5 (5.00%) high mild
9 (9.00%) high severe
alt_bn128_add_cdetrio6 time: [486.90 ns 492.22 ns 499.17 ns]
change: [+15.951% +18.859% +21.718%] (p = 0.00 < 0.05)
Performance has regressed.
alt_bn128_add_cdetrio7 time: [476.20 ns 477.32 ns 478.63 ns]
change: [+4.5727% +5.0589% +5.5639%] (p = 0.00 < 0.05)
Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) high mild
6 (6.00%) high severe
alt_bn128_add_cdetrio8 time: [469.91 ns 471.70 ns 473.86 ns]
change: [+3.4846% +4.1954% +4.8508%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
alt_bn128_add_cdetrio9 time: [462.73 ns 463.89 ns 465.18 ns]
change: [+0.7375% +1.7674% +2.6135%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
alt_bn128_add_cdetrio10 time: [462.89 ns 464.32 ns 466.09 ns]
change: [+1.1816% +1.7718% +2.3790%] (p = 0.00 < 0.05)
Performance has regressed.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) high mild
11 (11.00%) high severe
alt_bn128_add_cdetrio11 time: [7.0520 us 7.0686 us 7.0874 us]
change: [+1.4142% +2.3307% +3.2142%] (p = 0.00 < 0.05)
Performance has regressed.
Found 11 outliers among 100 measurements (11.00%)
3 (3.00%) high mild
8 (8.00%) high severe
alt_bn128_add_cdetrio12 time: [6.9087 us 6.9325 us 6.9617 us]
change: [+1.8834% +2.7203% +3.5571%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
alt_bn128_add_cdetrio13 time: [5.7721 us 5.7929 us 5.8161 us]
change: [+1.0361% +1.5660% +2.0925%] (p = 0.00 < 0.05)
Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
6 (6.00%) high mild
7 (7.00%) high severe
alt_bn128_add_cdetrio14 time: [1.1067 us 1.1110 us 1.1159 us]
change: [+1.5923% +2.1731% +2.7935%] (p = 0.00 < 0.05)
Performance has regressed.
Found 22 outliers among 100 measurements (22.00%)
11 (11.00%) low mild
5 (5.00%) high mild
6 (6.00%) high severe
alt_bn128_mul_chfast1 time: [47.973 us 49.006 us 50.165 us]
change: [+6.9874% +9.2457% +11.364%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
alt_bn128_mul_chfast2 time: [104.69 us 105.15 us 105.75 us]
change: [+6.0390% +8.3460% +10.755%] (p = 0.00 < 0.05)
Performance has regressed.
Found 20 outliers among 100 measurements (20.00%)
1 (1.00%) high mild
19 (19.00%) high severe
alt_bn128_mul_chfast3 time: [183.26 us 184.31 us 185.51 us]
change: [-3.6540% -2.0064% -0.5126%] (p = 0.01 < 0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
alt_bn128_mul_cdetrio1 time: [220.95 us 225.50 us 231.32 us]
change: [+18.269% +21.172% +24.188%] (p = 0.00 < 0.05)
Performance has regressed.
alt_bn128_mul_cdetrio6 time: [210.28 us 210.78 us 211.36 us]
change: [+2.2138% +2.7353% +3.2888%] (p = 0.00 < 0.05)
Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
7 (7.00%) high mild
6 (6.00%) high severe
alt_bn128_mul_cdetrio11 time: [226.19 us 232.66 us 240.37 us]
change: [+14.899% +17.286% +19.737%] (p = 0.00 < 0.05)
Performance has regressed.
alt_bn128_pairing_jeff1 time: [4.5175 ms 4.5245 ms 4.5326 ms]
change: [+2.0000% +2.6013% +3.1341%] (p = 0.00 < 0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
alt_bn128_pairing_jeff2 time: [4.5477 ms 4.5821 ms 4.6277 ms]
change: [+5.1775% +6.9971% +9.0209%] (p = 0.00 < 0.05)
Performance has regressed.
Found 19 outliers among 100 measurements (19.00%)
5 (5.00%) high mild
14 (14.00%) high severe
alt_bn128_pairing_jeff3 time: [4.5135 ms 4.5196 ms 4.5269 ms]
change: [+1.4982% +2.0518% +2.5441%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) high mild
4 (4.00%) high severe
alt_bn128_pairing_jeff4 time: [5.8452 ms 5.8984 ms 5.9696 ms]
change: [+1.4349% +1.9057% +2.4202%] (p = 0.00 < 0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
alt_bn128_pairing_jeff5 time: [6.1072 ms 6.1308 ms 6.1626 ms]
change: [+7.2239% +9.3211% +11.486%] (p = 0.00 < 0.05)
Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
2 (2.00%) high mild
15 (15.00%) high severe
alt_bn128_pairing_jeff6 time: [4.5186 ms 4.5485 ms 4.5872 ms]
change: [+2.2585% +2.6900% +3.2205%] (p = 0.00 < 0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
alt_bn128_pairing_empty_data
time: [29.119 ns 29.368 ns 29.681 ns]
change: [+2.7114% +3.5434% +4.6445%] (p = 0.00 < 0.05)
Performance has regressed.
Found 15 outliers among 100 measurements (15.00%)
5 (5.00%) high mild
10 (10.00%) high severe
alt_bn128_pairing_one_point
time: [3.0260 ms 3.0368 ms 3.0500 ms]
change: [-5.7801% -4.8809% -4.0162%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) high mild
6 (6.00%) high severe
alt_bn128_pairing_two_point_match_2
time: [3.6699 ms 3.7065 ms 3.7489 ms]
change: [-23.607% -21.653% -19.623%] (p = 0.00 < 0.05)
Performance has improved.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
alt_bn128_pairing_two_point_match_3
time: [4.6246 ms 4.7259 ms 4.8344 ms]
change: [-8.2545% -5.5461% -2.8819%] (p = 0.00 < 0.05)
Performance has improved.
Found 16 outliers among 100 measurements (16.00%)
5 (5.00%) high mild
11 (11.00%) high severe
alt_bn128_pairing_two_point_match_4
time: [4.7411 ms 4.7962 ms 4.8567 ms]
change: [+9.3491% +11.304% +13.421%] (p = 0.00 < 0.05)
Performance has regressed.
Found 21 outliers among 100 measurements (21.00%)
14 (14.00%) high mild
7 (7.00%) high severe
alt_bn128_pairing_ten_point_match_1
time: [14.606 ms 14.663 ms 14.741 ms]
change: [-8.9748% -7.6216% -6.3467%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
alt_bn128_pairing_ten_point_match_2
time: [15.554 ms 15.585 ms 15.628 ms]
change: [+0.1624% +0.5436% +0.9161%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high severe
alt_bn128_pairing_ten_point_match_3
time: [4.4080 ms 4.4138 ms 4.4200 ms]
change: [-0.7576% -0.1348% +0.3910%] (p = 0.66 > 0.05)
No change in performance detected.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment