sage: PGO instrumented compared to Release

gistfile1.txt

     Running benches/frontend.rs (target/x86_64-unknown-linux-gnu/release/deps/frontend-0e3d174564dee189)
Benchmarking GCC Backend/Hashmap (GCC precompiled run)
Benchmarking GCC Backend/Hashmap (GCC precompiled run): Warming up for 3.0000 s



Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 7.8s, enable flat sampling, or reduce sample count to 20.
Benchmarking GCC Backend/Hashmap (GCC precompiled run): Collecting 50 samples in estimated 7.8386 s (1275 iterations)
Benchmarking GCC Backend/Hashmap (GCC precompiled run): Analyzing
GCC Backend/Hashmap (GCC precompiled run)
                        time:   [6.1045 ms 6.1277 ms 6.1521 ms]
                        change: [-0.2609% +0.3943% +1.0860%] (p = 0.25 > 0.05)
                        No change in performance detected.
Found 1 outliers among 50 measurements (2.00%)
  1 (2.00%) high mild
Benchmarking GCC Backend/Vector (GCC precompiled run)


Benchmarking GCC Backend/Vector (GCC precompiled run): Warming up for 3.0000 s
Benchmarking GCC Backend/Vector (GCC precompiled run): Collecting 50 samples in estimated 5.5896 s (7650 iterations)
Benchmarking GCC Backend/Vector (GCC precompiled run): Analyzing
GCC Backend/Vector (GCC precompiled run)
                        time:   [722.13 µs 725.79 µs 728.70 µs]
                        change: [-0.9735% -0.2482% +0.3990%] (p = 0.49 > 0.05)
                        No change in performance detected.
Found 7 outliers among 50 measurements (14.00%)
  4 (8.00%) low severe
  3 (6.00%) low mild
Benchmarking GCC Backend/Vector2 (GCC precompiled run)


Benchmarking GCC Backend/Vector2 (GCC precompiled run): Warming up for 3.0000 s
Benchmarking GCC Backend/Vector2 (GCC precompiled run): Collecting 50 samples in estimated 5.5844 s (7650 iterations)
Benchmarking GCC Backend/Vector2 (GCC precompiled run): Analyzing
GCC Backend/Vector2 (GCC precompiled run)
                        time:   [725.91 µs 728.28 µs 730.18 µs]
                        change: [-0.6014% +0.1005% +0.7488%] (p = 0.78 > 0.05)
                        No change in performance detected.
Found 4 outliers among 50 measurements (8.00%)
  2 (4.00%) low severe
  2 (4.00%) low mild
Benchmarking GCC Backend/AES (GCC precompiled run)
Benchmarking GCC Backend/AES (GCC precompiled run): Warming up for 3.0000 s



Warning: Unable to complete 50 samples in 5.0s. You may wish to increase target time to 9.8s, enable flat sampling, or reduce sample count to 20.
Benchmarking GCC Backend/AES (GCC precompiled run): Collecting 50 samples in estimated 9.7861 s (1275 iterations)
Benchmarking GCC Backend/AES (GCC precompiled run): Analyzing
GCC Backend/AES (GCC precompiled run)
                        time:   [7.6131 ms 7.6419 ms 7.6697 ms]
                        change: [-0.5921% +0.2259% +1.0338%] (p = 0.59 > 0.05)
                        No change in performance detected.
Found 2 outliers among 50 measurements (4.00%)
  2 (4.00%) high mild
Benchmarking VM Backend/Hashmap (compile + VM run)

Benchmarking VM Backend/Hashmap (compile + VM run): Warming up for 3.0000 s
Benchmarking VM Backend/Hashmap (compile + VM run): Collecting 10 samples in estimated 6.2147 s (20 iterations)
Benchmarking VM Backend/Hashmap (compile + VM run): Analyzing
VM Backend/Hashmap (compile + VM run)
                        time:   [309.13 ms 310.48 ms 311.90 ms]
                        change: [+44.044% +44.885% +45.726%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking VM Backend/Vector (compile + VM run)
Benchmarking VM Backend/Vector (compile + VM run): Warming up for 3.0000 s
Benchmarking VM Backend/Vector (compile + VM run): Collecting 10 samples in estimated 7.2433 s (30 iterations)
Benchmarking VM Backend/Vector (compile + VM run): Analyzing
VM Backend/Vector (compile + VM run)
                        time:   [240.91 ms 241.29 ms 241.65 ms]
                        change: [+59.529% +59.781% +60.037%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) low mild
Benchmarking VM Backend/Vector2 (compile + VM run)
Benchmarking VM Backend/Vector2 (compile + VM run): Warming up for 3.0000 s
Benchmarking VM Backend/Vector2 (compile + VM run): Collecting 10 samples in estimated 7.6980 s (20 iterations)
Benchmarking VM Backend/Vector2 (compile + VM run): Analyzing
VM Backend/Vector2 (compile + VM run)
                        time:   [384.75 ms 385.34 ms 385.83 ms]
                        change: [+85.632% +86.065% +86.416%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 2 outliers among 10 measurements (20.00%)
  2 (20.00%) low mild
Benchmarking VM Backend/AES (compile + VM run)
Benchmarking VM Backend/AES (compile + VM run): Warming up for 3.0000 s

Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 9.0s.
Benchmarking VM Backend/AES (compile + VM run): Collecting 10 samples in estimated 9.0174 s (10 iterations)
Benchmarking VM Backend/AES (compile + VM run): Analyzing
VM Backend/AES (compile + VM run)
                        time:   [895.10 ms 901.29 ms 907.33 ms]
                        change: [+60.453% +62.513% +64.089%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking VM Backend/Hashmap (compile)
Benchmarking VM Backend/Hashmap (compile): Warming up for 3.0000 s
Benchmarking VM Backend/Hashmap (compile): Collecting 10 samples in estimated 6.4995 s (30 iterations)
Benchmarking VM Backend/Hashmap (compile): Analyzing
VM Backend/Hashmap (compile)
                        time:   [216.00 ms 216.34 ms 216.80 ms]
                        change: [+69.102% +69.427% +69.807%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking VM Backend/Vector (compile)
Benchmarking VM Backend/Vector (compile): Warming up for 3.0000 s
Benchmarking VM Backend/Vector (compile): Collecting 10 samples in estimated 7.2030 s (30 iterations)
Benchmarking VM Backend/Vector (compile): Analyzing
VM Backend/Vector (compile)
                        time:   [239.94 ms 240.19 ms 240.43 ms]
                        change: [+60.201% +60.401% +60.596%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking VM Backend/Vector2 (compile)
Benchmarking VM Backend/Vector2 (compile): Warming up for 3.0000 s
Benchmarking VM Backend/Vector2 (compile): Collecting 10 samples in estimated 7.6999 s (20 iterations)
Benchmarking VM Backend/Vector2 (compile): Analyzing
VM Backend/Vector2 (compile)
                        time:   [383.68 ms 384.44 ms 385.17 ms]
                        change: [+85.256% +85.998% +86.629%] (p = 0.00 < 0.05)
                        Performance has regressed.
Benchmarking VM Backend/AES (compile)
Benchmarking VM Backend/AES (compile): Warming up for 3.0000 s

Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 7.8s.
Benchmarking VM Backend/AES (compile): Collecting 10 samples in estimated 7.7536 s (10 iterations)
Benchmarking VM Backend/AES (compile): Analyzing
VM Backend/AES (compile)
                        time:   [773.42 ms 774.92 ms 776.40 ms]
                        change: [+71.023% +72.562% +73.550%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 3 outliers among 10 measurements (30.00%)
  2 (20.00%) low mild
  1 (10.00%) high mild
Benchmarking VM Backend/Hashmap (VM precompiled run)
Benchmarking VM Backend/Hashmap (VM precompiled run): Warming up for 3.0000 s
Benchmarking VM Backend/Hashmap (VM precompiled run): Collecting 10 samples in estimated 9.4752 s (110 iterations)
Benchmarking VM Backend/Hashmap (VM precompiled run): Analyzing
VM Backend/Hashmap (VM precompiled run)
                        time:   [87.297 ms 87.335 ms 87.372 ms]
                        change: [+0.8035% +1.4256% +2.0142%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) low mild
Benchmarking VM Backend/Vector (VM precompiled run)
Benchmarking VM Backend/Vector (VM precompiled run): Warming up for 3.0000 s
Benchmarking VM Backend/Vector (VM precompiled run): Collecting 10 samples in estimated 5.0188 s (3465 iterations)
Benchmarking VM Backend/Vector (VM precompiled run): Analyzing
VM Backend/Vector (VM precompiled run)
                        time:   [1.4539 ms 1.4545 ms 1.4551 ms]
                        change: [+1.0201% +1.6175% +2.2425%] (p = 0.00 < 0.05)
                        Performance has regressed.
Found 1 outliers among 10 measurements (10.00%)
  1 (10.00%) high mild
Benchmarking VM Backend/Vector2 (VM precompiled run)
Benchmarking VM Backend/Vector2 (VM precompiled run): Warming up for 3.0000 s
Benchmarking VM Backend/Vector2 (VM precompiled run): Collecting 10 samples in estimated 5.0260 s (3575 iterations)
Benchmarking VM Backend/Vector2 (VM precompiled run): Analyzing
VM Backend/Vector2 (VM precompiled run)
                        time:   [1.4322 ms 1.4323 ms 1.4324 ms]
                        change: [+0.4608% +1.0243% +1.5685%] (p = 0.00 < 0.05)
                        Change within noise threshold.
Found 3 outliers among 10 measurements (30.00%)
  1 (10.00%) low severe
  1 (10.00%) high mild
  1 (10.00%) high severe
Benchmarking VM Backend/AES (VM precompiled run)
Benchmarking VM Backend/AES (VM precompiled run): Warming up for 3.0000 s

Warning: Unable to complete 10 samples in 5.0s. You may wish to increase target time to 5.7s or enable flat sampling.
Benchmarking VM Backend/AES (VM precompiled run): Collecting 10 samples in estimated 5.6554 s (55 iterations)
Benchmarking VM Backend/AES (VM precompiled run): Analyzing
VM Backend/AES (VM precompiled run)
                        time:   [102.03 ms 102.86 ms 103.58 ms]
                        change: [-1.2969% -0.4067% +0.3635%] (p = 0.38 > 0.05)
                        No change in performance detected.