Skip to content

Instantly share code, notes, and snippets.

@CertainLach

CertainLach/benchmarks.md Secret

Created April 27, 2024 00:13
Show Gist options
  • Save CertainLach/7d0bbd0d0272bdc5091e5071e20327ee to your computer and use it in GitHub Desktop.
Save CertainLach/7d0bbd0d0272bdc5091e5071e20327ee to your computer and use it in GitHub Desktop.
Download ZIP
Raw
benchmarks.md

Real world

Graalvm CI

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, takes longer than a hour

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 94.3 ± 0.6 93.3 96.3 1.00
Rust (alternative, rsjsonnet) 266.8 ± 4.9 260.3 279.4 2.83 ± 0.06

Kube-prometheus manifests

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 130.0 ± 1.4 127.1 132.3 1.00
Rust (alternative, rsjsonnet) 205.8 ± 5.4 198.8 216.9 1.58 ± 0.05

Benchmarks from C++ jsonnet (/perf_tests)

Large string join

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 5.6 ± 0.1 5.3 6.4 1.00
Rust (alternative, rsjsonnet) 45.8 ± 1.2 44.0 51.4 8.14 ± 0.30
Go 97.3 ± 1.4 94.8 100.8 17.30 ± 0.51
C++ 78.8 ± 1.6 75.7 83.2 14.01 ± 0.46

Large string template

Note: No results for Go, fails with os stack size exhausion

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 6.8 ± 0.2 6.3 7.5 1.00
Rust (alternative, rsjsonnet) 7.3 ± 0.2 6.8 8.1 1.06 ± 0.04

Realistic 1

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 12.7 ± 0.2 12.3 14.0 1.00
Rust (alternative, rsjsonnet) 50.1 ± 1.2 47.5 51.8 3.95 ± 0.11

Realistic 2

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 262.9 ± 1.7 259.1 266.6 1.00
Rust (alternative, rsjsonnet) 548.5 ± 8.2 531.4 562.4 2.09 ± 0.03

Benchmarks from C++ jsonnet (/benchmarks)

Tail call

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 1.9 ± 0.1 1.6 2.3 1.00
Rust (alternative, rsjsonnet) 3.1 ± 0.1 2.8 4.1 1.69 ± 0.12
Go 5.5 ± 0.2 4.9 6.6 2.99 ± 0.22
C++ 32.7 ± 0.3 31.9 34.2 17.61 ± 1.07

Inheritance recursion

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 381.9 ± 3.6 375.6 390.7 1.00
Rust (alternative, rsjsonnet) 497.7 ± 17.1 471.8 530.0 1.30 ± 0.05
Scala 421.5 ± 5.0 414.1 436.6 1.10 ± 0.02

Simple recursive call

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 82.4 ± 0.8 80.8 84.2 1.00
Rust (alternative, rsjsonnet) 203.1 ± 9.3 183.5 209.2 2.47 ± 0.12
C++ 196.1 ± 1.6 193.1 199.6 2.38 ± 0.03

Foldl string concat

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 8.9 ± 0.2 8.6 9.6 1.00
Rust (alternative, rsjsonnet) 311.6 ± 4.1 304.3 319.4 34.82 ± 0.78
Go 51.9 ± 3.4 49.2 76.4 5.80 ± 0.39

Array sorts

Note: No results for Scala, std.reverse is not implemented

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 3.2 ± 0.1 2.9 3.7 1.00
Rust (alternative, rsjsonnet) 14.6 ± 0.6 13.4 16.4 4.61 ± 0.23
Go 12.3 ± 0.2 11.5 13.2 3.89 ± 0.15

Lazy array

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 55.1 ± 1.1 52.6 58.3 1.30 ± 0.04
Rust (alternative, rsjsonnet) 42.4 ± 1.0 40.8 47.2 1.00
C++ 187.9 ± 1.7 185.3 191.7 4.43 ± 0.11

Inheritance function recursion

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 1.6 ± 0.1 1.4 2.1 1.00
Rust (alternative, rsjsonnet) 2.9 ± 0.1 2.6 3.4 1.79 ± 0.12
Go 2.5 ± 0.2 2.1 3.1 1.55 ± 0.13

String strips

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 8.7 ± 0.1 8.5 9.1 1.00
Rust (alternative, rsjsonnet) 18.6 ± 0.5 17.6 19.8 2.13 ± 0.06
Go 11.4 ± 0.3 10.6 13.6 1.31 ± 0.04

Big object

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 2.2 ± 0.1 2.0 2.7 1.00
Rust (alternative, rsjsonnet) 3.3 ± 0.1 2.9 4.0 1.50 ± 0.09
Go 4.2 ± 0.2 3.8 5.6 1.94 ± 0.13
C++ 29.1 ± 0.6 28.4 32.9 13.44 ± 0.67

Benchmarks from Go jsonnet (builtins)

std.base64

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 2.7 ± 0.1 2.5 3.2 1.00
Rust (alternative, rsjsonnet) 2292.4 ± 45.8 2186.2 2333.9 836.71 ± 41.05
Go 17.3 ± 0.4 16.3 19.2 6.31 ± 0.32

std.base64Decode

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 2.6 ± 0.1 2.3 3.2 1.00
Rust (alternative, rsjsonnet) 8047.7 ± 117.9 7831.7 8248.2 3096.65 ± 148.68
Go 15.5 ± 0.4 14.7 18.0 5.96 ± 0.32

std.base64DecodeBytes

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 35.3 ± 0.4 34.5 36.5 1.00
Rust (alternative, rsjsonnet) 8409.5 ± 96.6 8278.5 8641.0 238.48 ± 3.96

std.base64 (byte array)

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 30.6 ± 0.4 29.8 32.5 1.00
Rust (alternative, rsjsonnet) 1513.4 ± 43.2 1417.2 1558.2 49.45 ± 1.56

std.foldl

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 5.3 ± 0.1 5.0 5.9 1.00
Rust (alternative, rsjsonnet) 82.8 ± 2.1 79.9 87.5 15.66 ± 0.56
Go 27.3 ± 0.8 25.7 31.8 5.16 ± 0.20
C++ 269.6 ± 4.2 264.5 276.7 50.98 ± 1.49

std.manifestJsonEx

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 1.7 ± 0.1 1.4 2.1 1.00
Rust (alternative, rsjsonnet) 2.9 ± 0.1 2.6 3.4 1.75 ± 0.12
Go 3.3 ± 0.2 2.9 4.5 2.01 ± 0.17

std.manifestTomlEx

Note: No results for Scala, std.manifestTomlEx is not implemented

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 1.7 ± 0.1 1.4 2.1 1.00
Rust (alternative, rsjsonnet) 7.9 ± 0.3 7.3 8.6 4.78 ± 0.30
Go 3.4 ± 0.2 2.8 4.6 2.03 ± 0.16

std.parseInt

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 1.7 ± 0.1 1.4 2.2 1.00
Rust (alternative, rsjsonnet) 2.9 ± 0.1 2.6 3.4 1.77 ± 0.12
Go 2.9 ± 0.2 2.4 3.5 1.75 ± 0.15

std.reverse

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, std.reverse is not implemented

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 46.0 ± 0.6 44.8 47.1 1.00
Rust (alternative, rsjsonnet) 810.2 ± 19.8 770.5 837.4 17.60 ± 0.48

std.substr

Note: No results for Scala, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 2.1 ± 0.1 1.9 2.8 1.00
Rust (alternative, rsjsonnet) 3.3 ± 0.1 3.1 3.9 1.53 ± 0.10
Go 7.0 ± 0.3 6.2 7.7 3.28 ± 0.21
C++ 29.4 ± 0.3 28.7 30.5 13.83 ± 0.76

Comparsion for array

Note: No results for Scala, array comparsion is not implemented

Note: No results for C++, slow benchmark, but only quick requested

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 25.9 ± 0.2 25.4 26.7 1.00
Rust (alternative, rsjsonnet) 234.6 ± 6.0 223.0 241.3 9.04 ± 0.25
Go 143.2 ± 9.1 130.9 166.6 5.52 ± 0.35

Comparsion for primitives

Note: No results for Go, slow benchmark, but only quick requested

Note: No results for Scala, slow benchmark, but only quick requested

Note: No results for C++, can't run: uses up to 192GB of RAM

Command Mean [ms] Min [ms] Max [ms] Relative
Rust 393.7 ± 6.1 382.7 404.6 1.00
Rust (alternative, rsjsonnet) 1358.0 ± 30.4 1303.2 1444.8 3.45 ± 0.09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment