Skip to content

Instantly share code, notes, and snippets.

@hero78119

hero78119/supernova benchmark.md

Last active August 17, 2023 08:11
Show Gist options
  • Save hero78119/5e955ddd51135b12a7bf26f54148dd17 to your computer and use it in GitHub Desktop.
Save hero78119/5e955ddd51135b12a7bf26f54148dd17 to your computer and use it in GitHub Desktop.
Download ZIP
supernova benchmark.md
Raw
supernova benchmark.md

Env

  • Under Windows-Subsystem-Linux 2.0, Ubuntu 22.04.1 LTS
  • CPU: AMD Ryzen 5 3600 6-Core Processor
  • RAM: 16GB
  • Based on commit: 500039d3658938dedb321082706779063d541945

Nova cargo criterion --features supernova --bench "recursive-snark"

RecursiveSNARK-StepCircuitSize-0/Prove
                        time:   [88.585 ms 89.706 ms 90.573 ms]
                        change: [-2.8517% -1.0769% +0.7450%] (p = 0.27 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-0/Verify
                        time:   [39.794 ms 40.090 ms 40.579 ms]
                        change: [-1.2288% -0.4598% +0.5307%] (p = 0.32 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-6565/Prove
                        time:   [99.341 ms 100.46 ms 101.68 ms]
                        change: [-1.2498% +0.0676% +1.3373%] (p = 0.92 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-6565/Verify
                        time:   [49.045 ms 49.409 ms 49.791 ms]
                        change: [-0.6980% +0.8305% +3.1726%] (p = 0.54 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-22949/Prove
                        time:   [125.75 ms 127.36 ms 128.75 ms]
                        change: [-1.2694% +1.0780% +3.9063%] (p = 0.52 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-22949/Verify
                        time:   [72.099 ms 72.550 ms 73.008 ms]
                        change: [-6.2177% -4.7820% -3.0518%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-55717/Prove
                        time:   [178.72 ms 179.94 ms 181.48 ms]
                        change: [-6.0436% -4.6645% -3.2626%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-55717/Verify
                        time:   [121.07 ms 121.50 ms 121.89 ms]
                        change: [-5.3224% -4.2157% -3.0807%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-121253/Prove
                        time:   [287.36 ms 290.69 ms 294.96 ms]
                        change: [-8.0857% -6.7813% -5.2592%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-121253/Verify
                        time:   [212.00 ms 213.61 ms 215.30 ms]
                        change: [-6.0256% -4.8710% -3.8024%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-252325/Prove
                        time:   [522.24 ms 524.96 ms 528.12 ms]
                        change: [-11.975% -9.8391% -7.6696%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-252325/Verify
                        time:   [402.23 ms 405.60 ms 409.34 ms]
                        change: [-9.4659% -8.4145% -7.3386%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-514469/Prove
                        time:   [1.0096 s 1.0271 s 1.0458 s]
                        change: [-15.968% -13.928% -11.690%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-514469/Verify
                        time:   [826.47 ms 857.90 ms 895.03 ms]
                        change: [-11.790% -8.4390% -4.4190%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-1038757/Prove
                        time:   [2.0997 s 2.1244 s 2.1492 s]
RecursiveSNARK-StepCircuitSize-1038757/Verify
                        time:   [1.7408 s 1.7463 s 1.7523 s]

Supernova cargo criterion --features supernova --bench "recursive-snark-supernova"

RecursiveSNARK-StepCircuitSize-0/Prove
                        time:   [89.386 ms 92.566 ms 95.291 ms]
                        change: [-13.862% -11.857% -9.4479%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-0/Verify
                        time:   [52.797 ms 82.217 ms 106.52 ms]
                        change: [+1.2654% +46.122% +92.815%] (p = 0.07 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-6565/Prove
                        time:   [99.272 ms 99.590 ms 100.11 ms]
                        change: [-13.734% -11.857% -10.003%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-6565/Verify
                        time:   [49.796 ms 50.011 ms 50.236 ms]
                        change: [-7.0904% -6.3919% -5.6006%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-22949/Prove
                        time:   [125.74 ms 126.26 ms 127.03 ms]
                        change: [-9.7094% -8.6383% -7.4431%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-22949/Verify
                        time:   [76.966 ms 77.184 ms 77.371 ms]
                        change: [-5.7859% -3.9966% -2.7310%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-55717/Prove
                        time:   [184.83 ms 185.59 ms 186.44 ms]
                        change: [-6.6272% -5.7749% -5.0198%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-55717/Verify
                        time:   [125.29 ms 125.79 ms 126.35 ms]
                        change: [-5.3828% -4.8038% -4.2198%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-121253/Prove
                        time:   [305.14 ms 309.40 ms 315.06 ms]
                        change: [-8.1006% -6.7225% -5.0663%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-121253/Verify
                        time:   [217.86 ms 218.85 ms 220.17 ms]
                        change: [-10.828% -9.8358% -8.8517%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-252325/Prove
                        time:   [546.95 ms 561.31 ms 578.58 ms]
                        change: [-11.791% -9.1563% -6.1481%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-252325/Verify
                        time:   [429.47 ms 433.05 ms 438.33 ms]
                        change: [-11.679% -10.022% -8.1894%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-514469/Prove
                        time:   [1.1889 s 1.2040 s 1.2204 s]
                        change: [-2.3928% -0.7165% +0.9608%] (p = 0.45 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-514469/Verify
                        time:   [913.19 ms 921.09 ms 929.15 ms]
                        change: [-4.5366% -3.5200% -2.5016%] (p = 0.00 < 0.05)
                        Performance has improved.
RecursiveSNARK-StepCircuitSize-1038757/Prove
                        time:   [1.8093 s 1.8717 s 1.9390 s]
                        change: [-7.0766% -3.4009% +0.7168%] (p = 0.13 > 0.05)
                        No change in performance detected.
RecursiveSNARK-StepCircuitSize-1038757/Verify
                        time:   [1.5348 s 1.5436 s 1.5521 s]
                        change: [-3.3115% -2.6087% -1.9320%] (p = 0.00 < 0.05)
                        Performance has improved.
@wyattbenno777
Copy link

Do you know if there is a way to see the space efficiency between the two? Do they take about the same amount of memory to run or is SuperNova much bigger?

@hero78119
Copy link
Author

Do you know if there is a way to see the space efficiency between the two? Do they take about the same amount of memory to run or is SuperNova much bigger?

Hi, from my observation there is no significant different under single circuit. More accurate result need to have memory profiling

@hero78119
Copy link
Author

hero78119 commented Aug 16, 2023 •
edited

Env

- Under Windows-Subsystem-Linux 2.0, Ubuntu 22.04.1 LTS
- CPU: AMD Ryzen 5 3600 6-Core Processor
- RAM: 16GB
- commits: https://github.com/microsoft/Nova/pull/204/commits/f76dfc2d6276b50272feaeaae57e0f35dffe5838

image

nova cargo criterion --bench recursive-snark

RecursiveSNARK-StepCircuitSize-0/Prove
                        time:   [63.941 ms 64.838 ms 65.548 ms]
RecursiveSNARK-StepCircuitSize-0/Verify
                        time:   [39.525 ms 39.747 ms 39.960 ms]
RecursiveSNARK-StepCircuitSize-6565/Prove
                        time:   [75.530 ms 76.028 ms 76.738 ms]
RecursiveSNARK-StepCircuitSize-6565/Verify
                        time:   [48.726 ms 48.978 ms 49.340 ms]
RecursiveSNARK-StepCircuitSize-22949/Prove
                        time:   [101.08 ms 102.35 ms 103.40 ms]
RecursiveSNARK-StepCircuitSize-22949/Verify
                        time:   [71.867 ms 72.123 ms 72.429 ms]
RecursiveSNARK-StepCircuitSize-55717/Prove
                        time:   [150.91 ms 152.10 ms 153.12 ms]
RecursiveSNARK-StepCircuitSize-55717/Verify
                        time:   [120.22 ms 120.71 ms 121.19 ms]
RecursiveSNARK-StepCircuitSize-121253/Prove
                        time:   [139.93 ms 140.90 ms 141.62 ms]
RecursiveSNARK-StepCircuitSize-121253/Verify
                        time:   [80.122 ms 80.467 ms 81.016 ms]
RecursiveSNARK-StepCircuitSize-252325/Prove
                        time:   [168.87 ms 170.05 ms 170.90 ms]
RecursiveSNARK-StepCircuitSize-252325/Verify
                        time:   [104.80 ms 105.14 ms 105.71 ms]
RecursiveSNARK-StepCircuitSize-514469/Prove
                        time:   [239.23 ms 241.69 ms 245.53 ms]
RecursiveSNARK-StepCircuitSize-514469/Verify
                        time:   [158.78 ms 159.31 ms 159.80 ms]
RecursiveSNARK-StepCircuitSize-1038757/Prove
                        time:   [359.33 ms 361.03 ms 362.59 ms]
RecursiveSNARK-StepCircuitSize-1038757/Verify
                        time:   [255.23 ms 258.92 ms 262.67 ms]

supernova cargo criterion --bench recursive-snark-supernova --features="supernova"

RecursiveSNARKSuperNova-1circuit-StepCircuitSize-0/Prove
                        time:   [64.560 ms 64.814 ms 65.100 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-0/Verify
                        time:   [39.525 ms 39.612 ms 39.754 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-6565/Prove
                        time:   [76.024 ms 76.428 ms 76.820 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-6565/Verify
                        time:   [48.812 ms 49.042 ms 49.257 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-22949/Prove
                        time:   [102.54 ms 103.72 ms 104.63 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-22949/Verify
                        time:   [74.342 ms 74.545 ms 74.803 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-55717/Prove
                        time:   [124.52 ms 124.86 ms 125.26 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-55717/Verify
                        time:   [66.108 ms 67.134 ms 68.138 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-121253/Prove
                        time:   [141.59 ms 142.02 ms 142.26 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-121253/Verify
                        time:   [81.448 ms 82.045 ms 82.688 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-252325/Prove
                        time:   [171.91 ms 172.90 ms 173.80 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-252325/Verify
                        time:   [108.69 ms 108.91 ms 109.06 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-514469/Prove
                        time:   [248.28 ms 249.96 ms 251.59 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-514469/Verify
                        time:   [165.75 ms 166.33 ms 167.13 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-1038757/Prove
                        time:   [400.24 ms 416.78 ms 441.51 ms]
RecursiveSNARKSuperNova-1circuit-StepCircuitSize-1038757/Verify
                        time:   [272.02 ms 274.93 ms 277.68 ms]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment