Skip to content

Instantly share code, notes, and snippets.

@jdimatteo
Created September 7, 2021 21:04
Show Gist options
  • Save jdimatteo/9087bac5aa8981b39f858b3a05f197e8 to your computer and use it in GitHub Desktop.
Save jdimatteo/9087bac5aa8981b39f858b3a05f197e8 to your computer and use it in GitHub Desktop.
(venv) jdimatteo@dimo:~/dev/great_expectations_fork_2$ py.test-benchmark compare --name long tests/performance/results/*.json --columns min
----------------------------------------------------------- benchmark: 144 tests -----------------------------------------------------------
Name (time in s) Min
--------------------------------------------------------------------------------------------------------------------------------------------
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V3] (initial_baseline_run_3) 3.2818 (1.0)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V3] (initial_baseline_run_2) 3.3227 (1.01)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V3] (initial_baseline_run_2) 3.5200 (1.07)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V2] (initial_baseline_run_2) 3.5963 (1.10)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V3] (minimal_multithreading_run_2) 3.6109 (1.10)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V3] (minimal_multithreading_run_3) 3.7080 (1.13)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V3] (initial_baseline_run_3) 3.8316 (1.17)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V3] (minimal_multithreading_run_3) 3.8629 (1.18)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V2] (initial_baseline_run_3) 3.8848 (1.18)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V3] (minimal_multithreading_run_3) 3.9193 (1.19)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V2] (initial_baseline_run_2) 3.9647 (1.21)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V3] (minimal_multithreading_run_1) 3.9875 (1.22)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V3] (minimal_multithreading_run_3) 4.0633 (1.24)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V3] (minimal_multithreading_run_1) 4.0660 (1.24)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V2] (minimal_multithreading_run_2) 4.1634 (1.27)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V2] (minimal_multithreading_run_3) 4.2091 (1.28)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V3] (initial_baseline_run_1) 4.2233 (1.29)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V3] (minimal_multithreading_run_2) 4.2280 (1.29)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V3] (minimal_multithreading_run_3) 4.3128 (1.31)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V3] (minimal_multithreading_run_3) 4.3367 (1.32)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V3] (initial_baseline_run_1) 4.3585 (1.33)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V3] (minimal_multithreading_run_2) 4.3846 (1.34)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V2] (minimal_multithreading_run_3) 4.4305 (1.35)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V2] (minimal_multithreading_run_3) 4.4564 (1.36)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V2] (initial_baseline_run_3) 4.4692 (1.36)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V3] (minimal_multithreading_run_1) 4.5081 (1.37)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V3] (minimal_multithreading_run_2) 4.5136 (1.38)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V3] (minimal_multithreading_run_1) 4.5222 (1.38)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V2] (minimal_multithreading_run_2) 4.5389 (1.38)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V2] (minimal_multithreading_run_3) 4.6477 (1.42)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V3] (minimal_multithreading_run_1) 4.6733 (1.42)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V3] (minimal_multithreading_run_3) 4.6950 (1.43)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V3] (minimal_multithreading_run_2) 4.7930 (1.46)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V2] (initial_baseline_run_1) 4.7968 (1.46)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V3] (minimal_multithreading_run_1) 4.7999 (1.46)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V2] (minimal_multithreading_run_2) 4.8708 (1.48)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V2] (minimal_multithreading_run_2) 4.8953 (1.49)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V2] (minimal_multithreading_run_2) 4.9247 (1.50)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V3] (minimal_multithreading_run_1) 4.9301 (1.50)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V2] (minimal_multithreading_run_1) 4.9308 (1.50)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V3] (minimal_multithreading_run_2) 4.9882 (1.52)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V3] (minimal_multithreading_run_2) 5.0461 (1.54)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V2] (minimal_multithreading_run_1) 5.0595 (1.54)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V2] (minimal_multithreading_run_3) 5.1363 (1.57)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V2] (minimal_multithreading_run_1) 5.1747 (1.58)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-False-V2] (initial_baseline_run_1) 5.2558 (1.60)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V3] (minimal_multithreading_run_1) 5.2768 (1.61)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V2] (minimal_multithreading_run_1) 5.3569 (1.63)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V2] (minimal_multithreading_run_1) 5.3841 (1.64)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V2] (minimal_multithreading_run_3) 5.5251 (1.68)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[1-True-V2] (minimal_multithreading_run_1) 5.5634 (1.70)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V2] (minimal_multithreading_run_1) 5.6320 (1.72)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V2] (minimal_multithreading_run_2) 5.7741 (1.76)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V3] (minimal_multithreading_run_3) 5.8482 (1.78)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V2] (minimal_multithreading_run_2) 6.0513 (1.84)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V2] (minimal_multithreading_run_3) 6.1236 (1.87)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V2] (minimal_multithreading_run_3) 6.5699 (2.00)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V3] (minimal_multithreading_run_1) 6.7213 (2.05)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V2] (minimal_multithreading_run_2) 6.8990 (2.10)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V3] (initial_baseline_run_2) 6.9249 (2.11)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V3] (minimal_multithreading_run_3) 7.0000 (2.13)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V2] (minimal_multithreading_run_2) 7.0039 (2.13)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V3] (initial_baseline_run_2) 7.0319 (2.14)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V2] (initial_baseline_run_2) 7.0574 (2.15)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V2] (minimal_multithreading_run_1) 7.1519 (2.18)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V3] (minimal_multithreading_run_2) 7.1914 (2.19)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V2] (minimal_multithreading_run_3) 7.2410 (2.21)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V2] (initial_baseline_run_3) 7.2650 (2.21)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V3] (minimal_multithreading_run_2) 7.4128 (2.26)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V2] (initial_baseline_run_2) 7.7079 (2.35)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V3] (initial_baseline_run_3) 7.8152 (2.38)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V2] (initial_baseline_run_3) 8.2846 (2.52)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V3] (initial_baseline_run_1) 8.3849 (2.55)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V3] (initial_baseline_run_1) 8.4396 (2.57)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V3] (initial_baseline_run_3) 8.4899 (2.59)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-False-V2] (initial_baseline_run_1) 8.7447 (2.66)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V2] (minimal_multithreading_run_1) 8.7775 (2.67)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[2-True-V2] (initial_baseline_run_1) 8.8481 (2.70)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V2] (minimal_multithreading_run_1) 10.1328 (3.09)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V2] (minimal_multithreading_run_3) 10.3947 (3.17)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V3] (minimal_multithreading_run_1) 10.8102 (3.29)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V2] (minimal_multithreading_run_2) 10.8611 (3.31)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V3] (minimal_multithreading_run_2) 11.1661 (3.40)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V3] (minimal_multithreading_run_3) 11.3730 (3.47)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V3] (initial_baseline_run_3) 13.0151 (3.97)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V3] (initial_baseline_run_2) 13.5262 (4.12)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V2] (initial_baseline_run_2) 13.7150 (4.18)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V3] (initial_baseline_run_3) 13.7633 (4.19)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V2] (initial_baseline_run_3) 14.2186 (4.33)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V3] (initial_baseline_run_2) 15.3856 (4.69)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V3] (initial_baseline_run_1) 15.4317 (4.70)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V2] (initial_baseline_run_3) 15.4466 (4.71)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V2] (initial_baseline_run_1) 16.2528 (4.95)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-True-V2] (initial_baseline_run_2) 16.5584 (5.05)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V3] (initial_baseline_run_1) 17.2043 (5.24)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[4-False-V2] (initial_baseline_run_1) 18.1744 (5.54)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V3] (minimal_multithreading_run_3) 22.3018 (6.80)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V2] (minimal_multithreading_run_3) 22.4979 (6.86)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V2] (minimal_multithreading_run_2) 23.3857 (7.13)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V3] (minimal_multithreading_run_1) 24.5691 (7.49)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V2] (minimal_multithreading_run_1) 25.5003 (7.77)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V3] (initial_baseline_run_3) 25.9606 (7.91)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V3] (minimal_multithreading_run_2) 26.3907 (8.04)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V2] (initial_baseline_run_3) 27.9552 (8.52)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V3] (initial_baseline_run_3) 28.0594 (8.55)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V3] (initial_baseline_run_2) 28.7145 (8.75)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V3] (initial_baseline_run_1) 29.2475 (8.91)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V2] (initial_baseline_run_3) 29.3288 (8.94)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V2] (initial_baseline_run_2) 29.4374 (8.97)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V3] (initial_baseline_run_1) 29.9335 (9.12)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V2] (initial_baseline_run_2) 30.3670 (9.25)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V3] (initial_baseline_run_2) 30.9665 (9.44)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-True-V2] (initial_baseline_run_1) 33.0360 (10.07)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[8-False-V2] (initial_baseline_run_1) 36.7383 (11.19)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V3] (initial_baseline_run_3) 52.4382 (15.98)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V2] (initial_baseline_run_3) 53.6786 (16.36)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V3] (initial_baseline_run_2) 53.7829 (16.39)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V3] (initial_baseline_run_3) 55.9569 (17.05)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V2] (initial_baseline_run_2) 56.1152 (17.10)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V2] (minimal_multithreading_run_2) 60.8031 (18.53)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V2] (initial_baseline_run_2) 60.8217 (18.53)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V2] (minimal_multithreading_run_3) 60.8511 (18.54)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V3] (initial_baseline_run_1) 61.3593 (18.70)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V2] (initial_baseline_run_3) 61.3706 (18.70)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V3] (initial_baseline_run_1) 61.6513 (18.79)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V2] (initial_baseline_run_1) 62.8486 (19.15)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V2] (minimal_multithreading_run_1) 62.8813 (19.16)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-False-V2] (initial_baseline_run_1) 68.7432 (20.95)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[16-True-V3] (initial_baseline_run_2) 69.8583 (21.29)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V3] (minimal_multithreading_run_3) 100.0210 (30.48)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V3] (minimal_multithreading_run_1) 102.3436 (31.18)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V3] (minimal_multithreading_run_2) 104.1340 (31.73)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V3] (initial_baseline_run_3) 323.4481 (98.56)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V3] (initial_baseline_run_2) 328.5219 (100.10)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V2] (initial_baseline_run_3) 344.7229 (105.04)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V2] (initial_baseline_run_2) 348.5321 (106.20)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V3] (initial_baseline_run_2) 382.3974 (116.52)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V2] (initial_baseline_run_2) 384.1241 (117.05)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V2] (initial_baseline_run_1) 384.9857 (117.31)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V3] (initial_baseline_run_3) 391.4854 (119.29)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V3] (initial_baseline_run_1) 391.8299 (119.39)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V2] (initial_baseline_run_3) 409.9862 (124.93)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-False-V2] (initial_baseline_run_1) 435.7602 (132.78)
tests/performance/test_bigquery_benchmarks.py::test_taxi_trips_benchmark[100-True-V3] (initial_baseline_run_1) 477.5333 (145.51)
--------------------------------------------------------------------------------------------------------------------------------------------
Legend:
Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
OPS: Operations Per Second, computed as 1 / Mean
(venv) jdimatteo@dimo:~/dev/great_expectations_fork_2$ git describe
0.13.32-36-g408d0be3b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment