sage: LTO compared to Release

gistfile1.txt

     Running benches/frontend.rs (target/release/deps/frontend-b8481846c8705864)


Benchmarking GCC Backend/Hashmap (GCC precompiled run): Warming up for 3.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 7.8s, enable flat sampling, or reduce sample count to 20.
GCC Backend/Hashmap (GCC precompiled run)
                        time:   [6.0549 ms 6.0764 ms 6.0976 ms]
                        change: [-1.2542% -0.6228% +0.0481%] (p = 0.07 > 0.05)
                        No change in performance detected.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high mild


GCC Backend/Vector (GCC precompiled run)
                        time:   [722.02 µs 724.23 µs 726.32 µs]
                        change: [-0.4932% -0.0831% +0.3127%] (p = 0.68 > 0.05)
                        No change in performance detected.
Found 1 outliers among 50 measurements (2.00%)
  1 (2.00%) low mild


GCC Backend/Vector2 (GCC precompiled run)
                        time:   [720.14 µs 722.77 µs 725.09 µs]
                        change: [-1.2831% -0.6096% +0.0755%] (p = 0.09 > 0.05)
                        No change in performance detected.
Found 6 outliers among 50 measurements (12.00%)
  5 (10.00%) low severe
  1 (2.00%) low mild


Benchmarking GCC Backend/AES (GCC precompiled run): Warming up for 3.0000 s
Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 9.8s, enable flat sampling, or reduce sample count to 20.
GCC Backend/AES (GCC precompiled run)
                        time:   [7.5837 ms 7.6169 ms 7.6503 ms]
                        change: [-0.8931% +0.0221% +1.1363%] (p = 0.96 > 0.05)
                        No change in performance detected.
Found 3 outliers among 50 measurements (6.00%)
  1 (2.00%) high mild
  2 (4.00%) high severe

VM Backend/Hashmap (compile + VM run)
                        time:   [207.88 ms 208.18 ms 208.67 ms]
                        change: [-3.2489% -2.8544% -2.4347%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
Benchmarking VM Backend/Vector (compile + VM run): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 7.3s or enable flat sampling.
VM Backend/Vector (compile + VM run)
                        time:   [131.59 ms 131.63 ms 131.66 ms]
                        change: [-12.892% -12.822% -12.750%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking VM Backend/Vector2 (compile + VM run): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.7s or enable flat sampling.
VM Backend/Vector2 (compile + VM run)
                        time:   [176.61 ms 177.27 ms 177.80 ms]
                        change: [-14.794% -14.573% -14.319%] (p = 0.00 < 0.05)
                        Performance has improved.
VM Backend/AES (compile + VM run)
                        time:   [494.94 ms 496.05 ms 496.79 ms]
                        change: [-11.517% -10.557% -9.9574%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) low severe
Benchmarking VM Backend/Hashmap (compile): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.0s or enable flat sampling.
VM Backend/Hashmap (compile)
                        time:   [109.27 ms 109.37 ms 109.58 ms]
                        change: [-14.439% -14.301% -14.141%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) high severe
Benchmarking VM Backend/Vector (compile): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 7.1s or enable flat sampling.
VM Backend/Vector (compile)
                        time:   [128.56 ms 128.76 ms 129.12 ms]
                        change: [-14.188% -14.049% -13.850%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) high mild
  1 (10.00%) high severe
Benchmarking VM Backend/Vector2 (compile): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.5s or enable flat sampling.
VM Backend/Vector2 (compile)
                        time:   [173.12 ms 173.15 ms 173.18 ms]
                        change: [-16.497% -16.202% -15.999%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high severe
VM Backend/AES (compile)
                        time:   [372.65 ms 372.83 ms 373.01 ms]
                        change: [-17.682% -16.976% -16.557%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) low mild
Benchmarking VM Backend/Hashmap (VM precompiled run): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.6s or enable flat sampling.
VM Backend/Hashmap (VM precompiled run)
                        time:   [99.825 ms 99.921 ms 99.980 ms]
                        change: [+15.181% +15.948% +16.647%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 10 measurements (20.00%)
  1 (10.00%) low mild
  1 (10.00%) high mild
VM Backend/Vector (VM precompiled run)
                        time:   [1.6293 ms 1.6298 ms 1.6307 ms]
                        change: [+13.119% +13.799% +14.504%] (p = 0.00 < 0.05)
                        Performance has regressed.
VM Backend/Vector2 (VM precompiled run)
                        time:   [1.6304 ms 1.6715 ms 1.6946 ms]
                        change: [+13.717% +15.568% +17.472%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking VM Backend/AES (VM precompiled run): Warming up for 3.0000 s
Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 6.6s or enable flat sampling.
VM Backend/AES (VM precompiled run)
                        time:   [120.02 ms 122.44 ms 123.55 ms]
                        change: [+15.339% +16.728% +18.144%] (p = 0.00 < 0.05)
                        Performance has regressed.