Skip to content

Instantly share code, notes, and snippets.

@vertexclique

vertexclique/reproducible_benchmarks

Created November 11, 2020 03:54
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save vertexclique/b5860ad836e78044d331cd3bb93fcf20 to your computer and use it in GitHub Desktop.
Save vertexclique/b5860ad836e78044d331cd3bb93fcf20 to your computer and use it in GitHub Desktop.
Download ZIP
ARROW-10500: reproducible benchmarks
Raw
reproducible_benchmarks
test result: ok. 0 passed; 0 failed; 501 ignored; 0 measured; 0 filtered out
Running /home/vertexclique/projects/arrow/rust/target/release/deps/aggregate_kernels-6d55c08b7328dd4e
sum 512 time: [631.40 ns 631.72 ns 632.14 ns]
change: [+0.0700% +0.1267% +0.1817%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 20 outliers among 100 measurements (20.00%)
1 (1.00%) low mild
19 (19.00%) high severe
min 512 time: [918.20 ns 918.24 ns 918.29 ns]
change: [+0.0011% +0.0152% +0.0289%] (p = 0.03 < 0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
sum nulls 512 time: [597.94 ns 598.19 ns 598.44 ns]
change: [+62.786% +62.851% +62.924%] (p = 0.00 < 0.05)
Performance has regressed.
Found 3 outliers among 100 measurements (3.00%)
3 (3.00%) high mild
min nulls 512 time: [2.5223 us 2.5228 us 2.5234 us]
change: [+8.3448% +8.4109% +8.4789%] (p = 0.00 < 0.05)
Performance has regressed.
Found 13 outliers among 100 measurements (13.00%)
4 (4.00%) high mild
9 (9.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/arithmetic_kernels-514094fccab7c0e2
add 512 time: [1.2554 us 1.2556 us 1.2558 us]
change: [-0.3519% -0.3191% -0.2912%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe
subtract 512 time: [1.3127 us 1.3130 us 1.3132 us]
change: [-10.649% -10.610% -10.582%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) low mild
5 (5.00%) high mild
4 (4.00%) high severe
multiply 512 time: [1.4736 us 1.4748 us 1.4767 us]
change: [+2.6484% +2.6966% +2.7539%] (p = 0.00 < 0.05)
Performance has regressed.
Found 6 outliers among 100 measurements (6.00%)
3 (3.00%) high mild
3 (3.00%) high severe
divide 512 time: [1.9068 us 1.9078 us 1.9088 us]
change: [-6.0314% -5.9703% -5.9156%] (p = 0.00 < 0.05)
Performance has improved.
limit 512, 512 time: [124.39 ns 124.40 ns 124.41 ns]
change: [-0.2092% -0.1876% -0.1669%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 5 outliers among 100 measurements (5.00%)
1 (1.00%) low mild
3 (3.00%) high mild
1 (1.00%) high severe
add_nulls_512 time: [1.3678 us 1.3680 us 1.3683 us]
change: [-4.9193% -4.8461% -4.7980%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
5 (5.00%) high mild
2 (2.00%) high severe
divide_nulls_512 time: [1.9350 us 1.9356 us 1.9362 us]
change: [-2.1864% -1.9373% -1.5141%] (p = 0.00 < 0.05)
Performance has improved.
Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/array_from_vec-c6c1616e32aeb910
array_from_vec 128 time: [452.59 ns 453.38 ns 454.12 ns]
change: [-8.4210% -8.1584% -7.8946%] (p = 0.00 < 0.05)
Performance has improved.
array_from_vec 256 time: [697.01 ns 703.56 ns 712.53 ns]
change: [-5.3332% -4.9608% -4.4838%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
array_from_vec 512 time: [1.1742 us 1.1756 us 1.1770 us]
change: [-8.1914% -8.0411% -7.8996%] (p = 0.00 < 0.05)
Performance has improved.
array_string_from_vec 128
time: [2.2028 us 2.2034 us 2.2040 us]
change: [-7.7671% -7.7335% -7.7020%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
array_string_from_vec 256
time: [3.3106 us 3.3113 us 3.3119 us]
change: [-2.8610% -2.6535% -2.2981%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low severe
2 (2.00%) high mild
4 (4.00%) high severe
array_string_from_vec 512
time: [5.5463 us 5.5526 us 5.5592 us]
change: [-2.9734% -2.8639% -2.7585%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
7 (7.00%) low mild
1 (1.00%) high mild
struct_array_from_vec 128
time: [4.0445 us 4.0458 us 4.0472 us]
change: [-9.1389% -9.0727% -9.0068%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
struct_array_from_vec 256
time: [6.2730 us 6.2746 us 6.2762 us]
change: [-1.1490% -1.0930% -1.0399%] (p = 0.00 < 0.05)
Performance has improved.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
struct_array_from_vec 512
time: [9.8178 us 9.8303 us 9.8532 us]
change: [-2.7386% -2.6109% -2.4234%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
struct_array_from_vec 1024
time: [16.790 us 16.795 us 16.800 us]
change: [-0.3224% -0.2051% -0.1200%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/boolean_kernels-4ca0af6d1333fee5
and time: [39.010 us 39.015 us 39.021 us]
change: [-0.3437% -0.2925% -0.2390%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
7 (7.00%) high mild
1 (1.00%) high severe
or time: [39.117 us 39.125 us 39.135 us]
change: [-0.6033% -0.4781% -0.3743%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
2 (2.00%) high mild
3 (3.00%) high severe
not time: [19.705 us 19.712 us 19.719 us]
change: [-0.2111% -0.1149% -0.0151%] (p = 0.02 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/buffer_bit_ops-4b34ce96e0d70b67
buffer_bit_ops and time: [1.9541 us 1.9570 us 1.9609 us]
change: [+980.13% +981.60% +983.41%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/builder-5d08968b1309f6ea
bench_primitive time: [737.80 us 737.86 us 737.91 us]
thrpt: [5.2936 GiB/s 5.2941 GiB/s 5.2945 GiB/s]
change:
time: [-0.1973% -0.1837% -0.1698%] (p = 0.00 < 0.05)
thrpt: [+0.1700% +0.1840% +0.1977%]
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low severe
1 (1.00%) high mild
Benchmarking bench_bool: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 9.9s, enable flat sampling, or reduce sample count to 40.
bench_bool time: [1.9599 ms 1.9619 ms 1.9651 ms]
thrpt: [254.44 MiB/s 254.86 MiB/s 255.12 MiB/s]
change:
time: [-24.355% -24.261% -24.071%] (p = 0.00 < 0.05)
thrpt: [+31.703% +32.033% +32.197%]
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
6 (6.00%) low mild
5 (5.00%) high mild
1 (1.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/cast_kernels-f44ca5bdd83e79ea
cast int32 to int32 512 time: [29.703 ns 29.710 ns 29.717 ns]
change: [-0.7997% -0.7669% -0.7347%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
4 (4.00%) high mild
2 (2.00%) high severe
cast int32 to uint32 512
time: [11.453 us 11.454 us 11.455 us]
change: [-3.8321% -3.8190% -3.8069%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe
cast int32 to float32 512
time: [11.550 us 11.550 us 11.551 us]
change: [-7.7855% -7.7265% -7.6329%] (p = 0.00 < 0.05)
Performance has improved.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
5 (5.00%) high mild
6 (6.00%) high severe
cast int32 to float64 512
time: [11.551 us 11.692 us 12.002 us]
change: [-6.6772% -6.2450% -4.9942%] (p = 0.00 < 0.05)
Performance has improved.
Found 10 outliers among 100 measurements (10.00%)
5 (5.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severe
cast int32 to int64 512 time: [11.419 us 11.421 us 11.424 us]
change: [-0.6997% -0.6818% -0.6659%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
2 (2.00%) low severe
1 (1.00%) low mild
4 (4.00%) high mild
4 (4.00%) high severe
cast float32 to int32 512
time: [11.149 us 11.150 us 11.151 us]
change: [-6.7670% -6.6872% -6.5436%] (p = 0.00 < 0.05)
Performance has improved.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
cast float64 to float32 512
time: [10.924 us 10.926 us 10.928 us]
change: [-13.442% -13.422% -13.402%] (p = 0.00 < 0.05)
Performance has improved.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
cast float64 to uint64 512
time: [11.256 us 11.257 us 11.259 us]
change: [-9.9688% -9.9445% -9.9214%] (p = 0.00 < 0.05)
Performance has improved.
Found 5 outliers among 100 measurements (5.00%)
2 (2.00%) high mild
3 (3.00%) high severe
cast int64 to int32 512 time: [12.043 us 12.044 us 12.045 us]
change: [+7.5900% +7.6099% +7.6297%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) low mild
2 (2.00%) high mild
cast date64 to date32 512
time: [20.284 us 20.287 us 20.290 us]
change: [-0.6720% -0.6170% -0.5789%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
cast date32 to date64 512
time: [20.026 us 20.028 us 20.030 us]
change: [-0.7292% -0.7029% -0.6812%] (p = 0.00 < 0.05)
Change within noise threshold.
cast time32s to time32ms 512
time: [2.0038 us 2.0041 us 2.0045 us]
change: [+6.2630% +6.3053% +6.3412%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
3 (3.00%) high mild
cast time32s to time64us 512
time: [13.837 us 13.839 us 13.841 us]
change: [+2.1580% +2.1853% +2.2099%] (p = 0.00 < 0.05)
Performance has regressed.
Found 17 outliers among 100 measurements (17.00%)
3 (3.00%) low severe
10 (10.00%) low mild
4 (4.00%) high mild
cast time64ns to time32s 512
time: [22.744 us 22.746 us 22.749 us]
change: [-0.0312% +0.0256% +0.1212%] (p = 0.62 > 0.05)
No change in performance detected.
Found 5 outliers among 100 measurements (5.00%)
5 (5.00%) high severe
cast timestamp_ns to timestamp_s 512
time: [31.760 ns 31.761 ns 31.762 ns]
change: [-0.0155% -0.0056% +0.0037%] (p = 0.26 > 0.05)
No change in performance detected.
Found 11 outliers among 100 measurements (11.00%)
1 (1.00%) low severe
2 (2.00%) high mild
8 (8.00%) high severe
cast timestamp_ms to timestamp_ns 512
time: [2.1720 us 2.1725 us 2.1730 us]
change: [+9.0141% +9.0499% +9.0881%] (p = 0.00 < 0.05)
Performance has regressed.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) low mild
1 (1.00%) high severe
cast timestamp_ms to i64 512
time: [176.05 ns 176.09 ns 176.12 ns]
change: [+0.3535% +0.3765% +0.3978%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 6 outliers among 100 measurements (6.00%)
1 (1.00%) low mild
5 (5.00%) high mild
Running /home/vertexclique/projects/arrow/rust/target/release/deps/comparison_kernels-3d1ca6a8af16bbbf
Benchmarking eq Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.6s, enable flat sampling, or reduce sample count to 60.
eq Float32 time: [1.2962 ms 1.2964 ms 1.2966 ms]
change: [+0.1254% +0.1763% +0.2209%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
4 (4.00%) high mild
5 (5.00%) high severe
Benchmarking eq scalar Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.2s, enable flat sampling, or reduce sample count to 60.
eq scalar Float32 time: [1.2173 ms 1.2175 ms 1.2177 ms]
change: [+0.1070% +0.1437% +0.1735%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
Benchmarking neq Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.2s, enable flat sampling, or reduce sample count to 60.
neq Float32 time: [1.2239 ms 1.2246 ms 1.2252 ms]
change: [+0.4366% +0.4915% +0.5475%] (p = 0.00 < 0.05)
Change within noise threshold.
Benchmarking neq scalar Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.2s, enable flat sampling, or reduce sample count to 60.
neq scalar Float32 time: [1.2185 ms 1.2186 ms 1.2187 ms]
change: [+0.1925% +0.2294% +0.2648%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Benchmarking lt Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.2s, enable flat sampling, or reduce sample count to 60.
lt Float32 time: [1.2183 ms 1.2188 ms 1.2193 ms]
change: [-0.1954% -0.0929% +0.0567%] (p = 0.21 > 0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high severe
Benchmarking lt scalar Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.2s, enable flat sampling, or reduce sample count to 60.
lt scalar Float32 time: [1.2209 ms 1.2212 ms 1.2214 ms]
change: [-0.2881% -0.2373% -0.1878%] (p = 0.00 < 0.05)
Change within noise threshold.
Benchmarking lt_eq Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.5s, enable flat sampling, or reduce sample count to 60.
lt_eq Float32 time: [1.2909 ms 1.2910 ms 1.2912 ms]
change: [+0.5075% +0.5280% +0.5503%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
2 (2.00%) high mild
2 (2.00%) high severe
Benchmarking lt_eq scalar Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.3s, enable flat sampling, or reduce sample count to 60.
lt_eq scalar Float32 time: [1.2450 ms 1.2451 ms 1.2453 ms]
change: [+0.1033% +0.1219% +0.1416%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) high mild
3 (3.00%) high severe
Benchmarking gt Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.4s, enable flat sampling, or reduce sample count to 60.
gt Float32 time: [1.2561 ms 1.2573 ms 1.2585 ms]
change: [+1.0780% +1.2364% +1.4017%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking gt scalar Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.0s, enable flat sampling, or reduce sample count to 60.
gt scalar Float32 time: [1.1902 ms 1.1905 ms 1.1908 ms]
change: [+0.2814% +0.3236% +0.3627%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
Benchmarking gt_eq Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.5s, enable flat sampling, or reduce sample count to 60.
gt_eq Float32 time: [1.2908 ms 1.2949 ms 1.3035 ms]
change: [+0.1943% +0.3174% +0.6117%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 3 outliers among 100 measurements (3.00%)
2 (2.00%) high mild
1 (1.00%) high severe
Benchmarking gt_eq scalar Float32: Warming up for 3.0000 s
Warning: Unable to complete 100 samples in 5.0s. You may wish to increase target time to 6.2s, enable flat sampling, or reduce sample count to 60.
gt_eq scalar Float32 time: [1.2196 ms 1.2197 ms 1.2199 ms]
change: [-2.4220% -2.3083% -2.1933%] (p = 0.00 < 0.05)
Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/csv_writer-7d66178e37754a56
record_batches_to_csv time: [67.250 us 67.371 us 67.526 us]
change: [+0.4845% +1.2742% +2.2805%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) low severe
3 (3.00%) low mild
1 (1.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/equal-199b566592734409
equal_512 time: [43.796 ns 43.799 ns 43.803 ns]
change: [+2.3308% +2.3443% +2.3566%] (p = 0.00 < 0.05)
Performance has regressed.
Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low severe
1 (1.00%) low mild
4 (4.00%) high mild
8 (8.00%) high severe
equal_nulls_512 time: [3.9190 us 3.9193 us 3.9197 us]
change: [+9.3227% +9.3585% +9.3886%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe
equal_string_512 time: [62.543 ns 62.550 ns 62.557 ns]
change: [-1.5828% -1.2098% -1.0005%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
4 (4.00%) high mild
4 (4.00%) high severe
equal_string_nulls_512 time: [4.5862 us 4.5885 us 4.5907 us]
change: [+7.3537% +7.4666% +7.6298%] (p = 0.00 < 0.05)
Performance has regressed.
Found 16 outliers among 100 measurements (16.00%)
1 (1.00%) low severe
4 (4.00%) low mild
10 (10.00%) high mild
1 (1.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/filter_kernels-9597d2fdb15818ab
filter u8 low selectivity
time: [138.75 us 138.81 us 138.91 us]
change: [+0.0243% +0.1306% +0.2860%] (p = 0.03 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
3 (3.00%) high mild
6 (6.00%) high severe
filter u8 high selectivity
time: [8.1861 us 8.1890 us 8.1914 us]
change: [+0.7762% +0.8167% +0.8573%] (p = 0.00 < 0.05)
Change within noise threshold.
filter u8 very low selectivity
time: [13.834 us 13.840 us 13.847 us]
change: [+0.2219% +0.2678% +0.3157%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild
filter context u8 low selectivity
time: [134.00 us 134.01 us 134.02 us]
change: [-0.1492% -0.1335% -0.1167%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
2 (2.00%) high mild
7 (7.00%) high severe
filter context u8 high selectivity
time: [3.2373 us 3.2378 us 3.2383 us]
change: [+0.0951% +0.2943% +0.4319%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 12 outliers among 100 measurements (12.00%)
5 (5.00%) low severe
2 (2.00%) low mild
3 (3.00%) high mild
2 (2.00%) high severe
filter context u8 very low selectivity
time: [8.7896 us 8.7907 us 8.7919 us]
change: [+0.2419% +0.2732% +0.3095%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
filter context u8 w NULLs low selectivity
time: [155.10 us 155.12 us 155.14 us]
change: [-0.0180% +0.2339% +0.6883%] (p = 0.25 > 0.05)
No change in performance detected.
Found 12 outliers among 100 measurements (12.00%)
1 (1.00%) low mild
1 (1.00%) high mild
10 (10.00%) high severe
filter context u8 w NULLs high selectivity
time: [3.6062 us 3.6085 us 3.6108 us]
change: [-0.7380% -0.6913% -0.6414%] (p = 0.00 < 0.05)
Change within noise threshold.
filter context u8 w NULLs very low selectivity
time: [155.24 us 155.30 us 155.35 us]
change: [+0.0074% +0.0531% +0.1005%] (p = 0.03 < 0.05)
Change within noise threshold.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high severe
filter context f32 low selectivity
time: [168.20 us 168.23 us 168.26 us]
change: [-0.0098% +0.0233% +0.0513%] (p = 0.14 > 0.05)
No change in performance detected.
Found 2 outliers among 100 measurements (2.00%)
1 (1.00%) high mild
1 (1.00%) high severe
filter context f32 high selectivity
time: [3.0048 us 3.0063 us 3.0078 us]
change: [+0.0778% +0.1936% +0.3188%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) low severe
3 (3.00%) low mild
1 (1.00%) high mild
3 (3.00%) high severe
filter context f32 very low selectivity
time: [21.514 us 21.518 us 21.521 us]
change: [-1.3767% -1.3494% -1.3229%] (p = 0.00 < 0.05)
Performance has improved.
Found 8 outliers among 100 measurements (8.00%)
7 (7.00%) high mild
1 (1.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/length_kernel-4a1d7c7d040d326a
length time: [39.828 us 39.834 us 39.839 us]
change: [+9.7146% +9.7397% +9.7643%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
1 (1.00%) low mild
6 (6.00%) high mild
Running /home/vertexclique/projects/arrow/rust/target/release/deps/sort_kernel-eeaea45614bd3d84
sort 2^10 time: [609.10 us 609.13 us 609.17 us]
change: [-0.4769% -0.4635% -0.4524%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 17 outliers among 100 measurements (17.00%)
2 (2.00%) low mild
4 (4.00%) high mild
11 (11.00%) high severe
sort 2^12 time: [2.9159 ms 2.9161 ms 2.9163 ms]
change: [-2.6692% -2.6565% -2.6448%] (p = 0.00 < 0.05)
Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
5 (5.00%) high mild
8 (8.00%) high severe
sort nulls 2^10 time: [638.88 us 638.92 us 638.97 us]
change: [+0.1496% +0.1779% +0.2006%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
2 (2.00%) low mild
5 (5.00%) high mild
2 (2.00%) high severe
sort nulls 2^12 time: [3.0715 ms 3.0717 ms 3.0719 ms]
change: [+0.1456% +0.1539% +0.1625%] (p = 0.00 < 0.05)
Change within noise threshold.
Found 9 outliers among 100 measurements (9.00%)
5 (5.00%) high mild
4 (4.00%) high severe
Running /home/vertexclique/projects/arrow/rust/target/release/deps/take_kernels-41f56ca318892320
take i32 512 time: [8.4306 us 8.4317 us 8.4328 us]
change: [+2.8438% +2.8779% +2.9048%] (p = 0.00 < 0.05)
Performance has regressed.
Found 4 outliers among 100 measurements (4.00%)
1 (1.00%) low mild
2 (2.00%) high mild
1 (1.00%) high severe
take i32 1024 time: [16.281 us 16.284 us 16.286 us]
change: [+3.4473% +3.5791% +3.6523%] (p = 0.00 < 0.05)
Performance has regressed.
Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) high mild
take bool 512 time: [8.2206 us 8.2230 us 8.2253 us]
change: [+2.6158% +2.7032% +2.8158%] (p = 0.00 < 0.05)
Performance has regressed.
Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) low severe
1 (1.00%) high mild
2 (2.00%) high severe
take bool 1024 time: [16.091 us 16.143 us 16.255 us]
change: [+2.9255% +3.2717% +3.7418%] (p = 0.00 < 0.05)
Performance has regressed.
Found 8 outliers among 100 measurements (8.00%)
2 (2.00%) low mild
5 (5.00%) high mild
1 (1.00%) high severe
take str 512 time: [79.094 us 79.166 us 79.245 us]
change: [+1.1497% +1.2797% +1.3925%] (p = 0.00 < 0.05)
Performance has regressed.
take str 1024 time: [209.46 us 209.49 us 209.53 us]
change: [+5.6396% +5.6769% +5.7265%] (p = 0.00 < 0.05)
Performance has regressed.
Found 10 outliers among 100 measurements (10.00%)
7 (7.00%) high mild
3 (3.00%) high severe
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment