Skip to content

Instantly share code, notes, and snippets.

@hrhino

hrhino/results

Last active August 25, 2018 02:35
Show Gist options
  • Save hrhino/556f3c080160fde1473338d25c8d238c to your computer and use it in GitHub Desktop.
Save hrhino/556f3c080160fde1473338d25c8d238c to your computer and use it in GitHub Desktop.
Download ZIP
Raw
results
(all results are from quick builds using 2.12.5 as starr, optimisations enabled)
# v2.12.6
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 494
[info] mean = 960.969 ±(99.9%) 4.224 ms/op
[info] Histogram, ms/op:
[info] [ 900.000, 925.000) = 29
[info] [ 925.000, 950.000) = 162
[info] [ 950.000, 975.000) = 182
[info] [ 975.000, 1000.000) = 81
[info] [1000.000, 1025.000) = 23
[info] [1025.000, 1050.000) = 10
[info] [1050.000, 1075.000) = 4
[info] [1075.000, 1100.000) = 2
[info] [1100.000, 1125.000) = 1
[info] [1125.000, 1150.000) = 0
[info] [1150.000, 1175.000) = 0
[info] Percentiles, ms/op:
[info] p(0.0000) = 911.213 ms/op
[info] p(50.0000) = 955.253 ms/op
[info] p(90.0000) = 994.574 ms/op
[info] p(95.0000) = 1017.119 ms/op
[info] p(99.0000) = 1061.421 ms/op
[info] p(99.9000) = 1115.685 ms/op
[info] p(99.9900) = 1115.685 ms/op
[info] p(99.9990) = 1115.685 ms/op
[info] p(99.9999) = 1115.685 ms/op
[info] p(100.0000) = 1115.685 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 299.403 ±(99.9%) 2.372 MB/sec [Average]
[info] (min, avg, max) = (285.450, 299.403, 308.276), stdev = 4.513
[info] CI (99.9%): [297.032, 301.775] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 316171170.188 ±(99.9%) 294979.124 B/op [Average]
[info] (min, avg, max) = (315474058.909, 316171170.188, 316909920.000), stdev = 561228.483
[info] CI (99.9%): [315876191.065, 316466149.312] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 301.636 ±(99.9%) 14.930 MB/sec [Average]
[info] (min, avg, max) = (266.348, 301.636, 339.435), stdev = 28.405
[info] CI (99.9%): [286.706, 316.566] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 318480897.849 ±(99.9%) 15265864.679 B/op [Average]
[info] (min, avg, max) = (283163182.545, 318480897.849, 352798161.455), stdev = 29044896.341
[info] CI (99.9%): [303215033.170, 333746762.528] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.177 ±(99.9%) 0.255 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.177, 2.404), stdev = 0.485
[info] CI (99.9%): [≈ 0, 0.432] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 188363.137 ±(99.9%) 271764.480 B/op [Average]
[info] (min, avg, max) = (≈ 0, 188363.137, 2569714.909), stdev = 517060.206
[info] CI (99.9%): [≈ 0, 460127.617] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 248.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.511, 6.000), stdev = 0.506
[info] CI (99.9%): [248.000, 248.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5515.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (62.000, 122.556, 148.000), stdev = 19.224
[info] CI (99.9%): [5515.000, 5515.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:42
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 494 960.969 ± 4.224 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 911.213 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 955.253 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 994.574 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 1017.119 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 1061.421 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 1115.685 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 1115.685 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 1115.685 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 299.403 ± 2.372 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 316171170.188 ± 294979.124 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 301.636 ± 14.930 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 318480897.849 ± 15265864.679 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 0.177 ± 0.255 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 188363.137 ± 271764.480 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 248.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-d7ae7a8-SNAPSHOT scalap sample 45 5515.000 ms
[success] Total time: 1005 s, completed Aug 23, 2018 8:15:45 PM
# reduce varargs allocation for appliedType
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 953.075 ±(99.9%) 3.857 ms/op
[info] Histogram, ms/op:
[info] [ 900.000, 912.500) = 5
[info] [ 912.500, 925.000) = 38
[info] [ 925.000, 937.500) = 113
[info] [ 937.500, 950.000) = 106
[info] [ 950.000, 962.500) = 79
[info] [ 962.500, 975.000) = 67
[info] [ 975.000, 987.500) = 38
[info] [ 987.500, 1000.000) = 21
[info] [1000.000, 1012.500) = 15
[info] [1012.500, 1025.000) = 3
[info] [1025.000, 1037.500) = 4
[info] [1037.500, 1050.000) = 3
[info] [1050.000, 1062.500) = 0
[info] [1062.500, 1075.000) = 3
[info] [1075.000, 1087.500) = 0
[info] Percentiles, ms/op:
[info] p(0.0000) = 904.921 ms/op
[info] p(50.0000) = 947.913 ms/op
[info] p(90.0000) = 987.129 ms/op
[info] p(95.0000) = 1002.648 ms/op
[info] p(99.0000) = 1046.479 ms/op
[info] p(99.9000) = 1072.693 ms/op
[info] p(99.9900) = 1072.693 ms/op
[info] p(99.9990) = 1072.693 ms/op
[info] p(99.9999) = 1072.693 ms/op
[info] p(100.0000) = 1072.693 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 302.665 ±(99.9%) 1.871 MB/sec [Average]
[info] (min, avg, max) = (295.398, 302.665, 310.128), stdev = 3.559
[info] CI (99.9%): [300.794, 304.535] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 317086553.600 ±(99.9%) 1045657.231 B/op [Average]
[info] (min, avg, max) = (315182992.727, 317086553.600, 319842005.091), stdev = 1989471.709
[info] CI (99.9%): [316040896.369, 318132210.831] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 302.677 ±(99.9%) 13.736 MB/sec [Average]
[info] (min, avg, max) = (264.410, 302.677, 333.076), stdev = 26.134
[info] CI (99.9%): [288.941, 316.413] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 317159287.467 ±(99.9%) 14659777.470 B/op [Average]
[info] (min, avg, max) = (281828631.273, 317159287.467, 347078656.000), stdev = 27891752.348
[info] CI (99.9%): [302499509.996, 331819064.937] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.433 ±(99.9%) 0.290 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.433, 2.410), stdev = 0.552
[info] CI (99.9%): [0.143, 0.723] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 454091.685 ±(99.9%) 304970.694 B/op [Average]
[info] (min, avg, max) = (≈ 0, 454091.685, 2517795.636), stdev = 580238.485
[info] CI (99.9%): [149120.991, 759062.379] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 250.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.556, 6.000), stdev = 0.503
[info] CI (99.9%): [250.000, 250.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5115.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (53.000, 113.667, 147.000), stdev = 23.987
[info] CI (99.9%): [5115.000, 5115.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:44
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 495 953.075 ± 3.857 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 904.921 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 947.913 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 987.129 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 1002.648 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 1046.479 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 1072.693 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 1072.693 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 1072.693 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 302.665 ± 1.871 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 317086553.600 ± 1045657.231 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 302.677 ± 13.736 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 317159287.467 ± 14659777.470 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 0.433 ± 0.290 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 454091.685 ± 304970.694 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 250.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-87cfaaf-SNAPSHOT scalap sample 45 5115.000 ms
[success] Total time: 1023 s, completed Aug 23, 2018 8:39:21 PM
# flatten changeOwner arguments
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 955.926 ±(99.9%) 4.027 ms/op
[info] Histogram, ms/op:
[info] [ 900.000, 925.000) = 37
[info] [ 925.000, 950.000) = 193
[info] [ 950.000, 975.000) = 169
[info] [ 975.000, 1000.000) = 61
[info] [1000.000, 1025.000) = 22
[info] [1025.000, 1050.000) = 9
[info] [1050.000, 1075.000) = 2
[info] [1075.000, 1100.000) = 1
[info] [1100.000, 1125.000) = 1
[info] [1125.000, 1150.000) = 0
[info] [1150.000, 1175.000) = 0
[info] Percentiles, ms/op:
[info] p(0.0000) = 914.358 ms/op
[info] p(50.0000) = 952.107 ms/op
[info] p(90.0000) = 990.695 ms/op
[info] p(95.0000) = 1006.633 ms/op
[info] p(99.0000) = 1049.708 ms/op
[info] p(99.9000) = 1107.296 ms/op
[info] p(99.9900) = 1107.296 ms/op
[info] p(99.9990) = 1107.296 ms/op
[info] p(99.9999) = 1107.296 ms/op
[info] p(100.0000) = 1107.296 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 300.120 ±(99.9%) 1.432 MB/sec [Average]
[info] (min, avg, max) = (293.256, 300.120, 306.264), stdev = 2.725
[info] CI (99.9%): [298.688, 301.552] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 315312247.935 ±(99.9%) 155014.318 B/op [Average]
[info] (min, avg, max) = (314966370.182, 315312247.935, 316384562.182), stdev = 294930.874
[info] CI (99.9%): [315157233.617, 315467262.254] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 301.418 ±(99.9%) 14.777 MB/sec [Average]
[info] (min, avg, max) = (266.691, 301.418, 334.958), stdev = 28.114
[info] CI (99.9%): [286.642, 316.195] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 316653978.602 ±(99.9%) 15324480.603 B/op [Average]
[info] (min, avg, max) = (280303429.818, 316653978.602, 350653346.909), stdev = 29156419.237
[info] CI (99.9%): [301329497.999, 331978459.205] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.292 ±(99.9%) 0.362 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.292, 2.559), stdev = 0.689
[info] CI (99.9%): [≈ 0, 0.654] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 306622.141 ±(99.9%) 380989.196 B/op [Average]
[info] (min, avg, max) = (≈ 0, 306622.141, 2659282.909), stdev = 724871.596
[info] CI (99.9%): [≈ 0, 687611.337] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 248.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.511, 6.000), stdev = 0.506
[info] CI (99.9%): [248.000, 248.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5684.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (83.000, 126.311, 147.000), stdev = 15.282
[info] CI (99.9%): [5684.000, 5684.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:40
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 495 955.926 ± 4.027 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 914.358 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 952.107 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 990.695 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 1006.633 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 1049.708 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 1107.296 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 1107.296 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 1107.296 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 300.120 ± 1.432 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 315312247.935 ± 155014.318 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 301.418 ± 14.777 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 316653978.602 ± 15324480.603 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 0.292 ± 0.362 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 306622.141 ± 380989.196 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 248.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-71073e4-SNAPSHOT scalap sample 45 5684.000 ms
[success] Total time: 1005 s, completed Aug 23, 2018 9:01:21 PM
# avoid some unnecessary list reversals
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 959.489 ±(99.9%) 3.971 ms/op
[info] Histogram, ms/op:
[info] [ 900.000, 912.500) = 2
[info] [ 912.500, 925.000) = 17
[info] [ 925.000, 937.500) = 83
[info] [ 937.500, 950.000) = 107
[info] [ 950.000, 962.500) = 88
[info] [ 962.500, 975.000) = 86
[info] [ 975.000, 987.500) = 52
[info] [ 987.500, 1000.000) = 25
[info] [1000.000, 1012.500) = 14
[info] [1012.500, 1025.000) = 6
[info] [1025.000, 1037.500) = 3
[info] [1037.500, 1050.000) = 7
[info] [1050.000, 1062.500) = 1
[info] [1062.500, 1075.000) = 2
[info] [1075.000, 1087.500) = 2
[info] Percentiles, ms/op:
[info] p(0.0000) = 907.018 ms/op
[info] p(50.0000) = 954.204 ms/op
[info] p(90.0000) = 990.275 ms/op
[info] p(95.0000) = 1007.891 ms/op
[info] p(99.0000) = 1055.329 ms/op
[info] p(99.9000) = 1082.130 ms/op
[info] p(99.9900) = 1082.130 ms/op
[info] p(99.9990) = 1082.130 ms/op
[info] p(99.9999) = 1082.130 ms/op
[info] p(100.0000) = 1082.130 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 299.950 ±(99.9%) 1.726 MB/sec [Average]
[info] (min, avg, max) = (294.596, 299.950, 307.179), stdev = 3.284
[info] CI (99.9%): [298.224, 301.676] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 316240392.453 ±(99.9%) 787948.713 B/op [Average]
[info] (min, avg, max) = (315341470.545, 316240392.453, 319776495.273), stdev = 1499154.432
[info] CI (99.9%): [315452443.740, 317028341.165] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 300.872 ±(99.9%) 14.520 MB/sec [Average]
[info] (min, avg, max) = (265.664, 300.872, 341.664), stdev = 27.626
[info] CI (99.9%): [286.352, 315.392] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 317234431.208 ±(99.9%) 15335593.081 B/op [Average]
[info] (min, avg, max) = (278492253.091, 317234431.208, 352607511.273), stdev = 29177561.883
[info] CI (99.9%): [301898838.127, 332570024.289] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.306 ±(99.9%) 0.291 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.306, 2.384), stdev = 0.554
[info] CI (99.9%): [0.015, 0.597] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 323119.903 ±(99.9%) 307452.978 B/op [Average]
[info] (min, avg, max) = (≈ 0, 323119.903, 2523235.636), stdev = 584961.289
[info] CI (99.9%): [15666.925, 630572.881] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 249.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.533, 6.000), stdev = 0.505
[info] CI (99.9%): [249.000, 249.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5252.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (53.000, 116.711, 148.000), stdev = 26.236
[info] CI (99.9%): [5252.000, 5252.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:49
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 495 959.489 ± 3.971 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 907.018 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 954.204 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 990.275 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 1007.891 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 1055.329 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 1082.130 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 1082.130 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 1082.130 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 299.950 ± 1.726 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 316240392.453 ± 787948.713 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 300.872 ± 14.520 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 317234431.208 ± 15335593.081 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 0.306 ± 0.291 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 323119.903 ± 307452.978 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 249.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-0913adc-SNAPSHOT scalap sample 45 5252.000 ms
[success] Total time: 1025 s, completed Aug 23, 2018 9:20:06 PM
# don't analyze feature traits every check
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 958.530 ±(99.9%) 4.438 ms/op
[info] Histogram, ms/op:
[info] [ 800.000, 825.000) = 0
[info] [ 825.000, 850.000) = 0
[info] [ 850.000, 875.000) = 0
[info] [ 875.000, 900.000) = 7
[info] [ 900.000, 925.000) = 41
[info] [ 925.000, 950.000) = 147
[info] [ 950.000, 975.000) = 178
[info] [ 975.000, 1000.000) = 90
[info] [1000.000, 1025.000) = 21
[info] [1025.000, 1050.000) = 5
[info] [1050.000, 1075.000) = 2
[info] [1075.000, 1100.000) = 2
[info] [1100.000, 1125.000) = 2
[info] [1125.000, 1150.000) = 0
[info] [1150.000, 1175.000) = 0
[info] Percentiles, ms/op:
[info] p(0.0000) = 891.290 ms/op
[info] p(50.0000) = 957.350 ms/op
[info] p(90.0000) = 993.001 ms/op
[info] p(95.0000) = 1007.891 ms/op
[info] p(99.0000) = 1059.817 ms/op
[info] p(99.9000) = 1121.976 ms/op
[info] p(99.9900) = 1121.976 ms/op
[info] p(99.9990) = 1121.976 ms/op
[info] p(99.9999) = 1121.976 ms/op
[info] p(100.0000) = 1121.976 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 300.201 ±(99.9%) 2.345 MB/sec [Average]
[info] (min, avg, max) = (293.009, 300.201, 313.906), stdev = 4.462
[info] CI (99.9%): [297.856, 302.546] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 316162614.174 ±(99.9%) 201564.628 B/op [Average]
[info] (min, avg, max) = (315581259.636, 316162614.174, 316734016.727), stdev = 383497.683
[info] CI (99.9%): [315961049.546, 316364178.802] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 299.901 ±(99.9%) 14.528 MB/sec [Average]
[info] (min, avg, max) = (265.284, 299.901, 346.421), stdev = 27.640
[info] CI (99.9%): [285.373, 314.428] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 315885069.689 ±(99.9%) 15328540.751 B/op [Average]
[info] (min, avg, max) = (278873553.455, 315885069.689, 348270219.636), stdev = 29164144.091
[info] CI (99.9%): [300556528.938, 331213610.440] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.231 ±(99.9%) 0.313 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.231, 2.502), stdev = 0.595
[info] CI (99.9%): [≈ 0, 0.544] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 244712.016 ±(99.9%) 333189.612 B/op [Average]
[info] (min, avg, max) = (≈ 0, 244712.016, 2613756.364), stdev = 633927.913
[info] CI (99.9%): [≈ 0, 577901.628] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 247.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.489, 6.000), stdev = 0.506
[info] CI (99.9%): [247.000, 247.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5701.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (85.000, 126.689, 157.000), stdev = 17.170
[info] CI (99.9%): [5701.000, 5701.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:47
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 495 958.530 ± 4.438 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 891.290 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 957.350 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 993.001 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 1007.891 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 1059.817 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 1121.976 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 1121.976 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 1121.976 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 300.201 ± 2.345 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 316162614.174 ± 201564.628 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 299.901 ± 14.528 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 315885069.689 ± 15328540.751 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 0.231 ± 0.313 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 244712.016 ± 333189.612 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 247.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-7e0bfea-SNAPSHOT scalap sample 45 5701.000 ms
[success] Total time: 1024 s, completed Aug 23, 2018 9:38:11 PM
# remove hoisted closures
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 961.883 ±(99.9%) 4.514 ms/op
[info] Histogram, ms/op:
[info] [ 900.000, 925.000) = 56
[info] [ 925.000, 950.000) = 105
[info] [ 950.000, 975.000) = 191
[info] [ 975.000, 1000.000) = 97
[info] [1000.000, 1025.000) = 28
[info] [1025.000, 1050.000) = 9
[info] [1050.000, 1075.000) = 5
[info] [1075.000, 1100.000) = 3
[info] [1100.000, 1125.000) = 1
[info] [1125.000, 1150.000) = 0
[info] [1150.000, 1175.000) = 0
[info] Percentiles, ms/op:
[info] p(0.0000) = 903.873 ms/op
[info] p(50.0000) = 957.350 ms/op
[info] p(90.0000) = 997.196 ms/op
[info] p(95.0000) = 1016.280 ms/op
[info] p(99.0000) = 1073.826 ms/op
[info] p(99.9000) = 1113.588 ms/op
[info] p(99.9900) = 1113.588 ms/op
[info] p(99.9990) = 1113.588 ms/op
[info] p(99.9999) = 1113.588 ms/op
[info] p(100.0000) = 1113.588 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 299.253 ±(99.9%) 2.370 MB/sec [Average]
[info] (min, avg, max) = (290.672, 299.253, 309.929), stdev = 4.508
[info] CI (99.9%): [296.884, 301.623] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 316214941.947 ±(99.9%) 600692.042 B/op [Average]
[info] (min, avg, max) = (315146177.455, 316214941.947, 318754687.273), stdev = 1142879.126
[info] CI (99.9%): [315614249.906, 316815633.989] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 299.886 ±(99.9%) 14.133 MB/sec [Average]
[info] (min, avg, max) = (264.961, 299.886, 338.239), stdev = 26.889
[info] CI (99.9%): [285.753, 314.018] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 316967578.117 ±(99.9%) 15233949.365 B/op [Average]
[info] (min, avg, max) = (281828631.273, 316967578.117, 353322449.455), stdev = 28984174.135
[info] CI (99.9%): [301733628.752, 332201527.482] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.303 ±(99.9%) 0.363 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.303, 2.477), stdev = 0.690
[info] CI (99.9%): [≈ 0, 0.666] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 319707.604 ±(99.9%) 382062.241 B/op [Average]
[info] (min, avg, max) = (≈ 0, 319707.604, 2598368.000), stdev = 726913.176
[info] CI (99.9%): [≈ 0, 701769.845] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 248.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.511, 6.000), stdev = 0.506
[info] CI (99.9%): [248.000, 248.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5415.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (46.000, 120.333, 148.000), stdev = 24.545
[info] CI (99.9%): [5415.000, 5415.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:45
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 495 961.883 ± 4.514 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 903.873 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 957.350 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 997.196 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 1016.280 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 1073.826 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 1113.588 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 1113.588 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 1113.588 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 299.253 ± 2.370 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 316214941.947 ± 600692.042 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 299.886 ± 14.133 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 316967578.117 ± 15233949.365 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 0.303 ± 0.363 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 319707.604 ± 382062.241 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 248.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-6479b95-SNAPSHOT scalap sample 45 5415.000 ms
[success] Total time: 1022 s, completed Aug 23, 2018 9:59:22 PM
# faster `sequence`
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 951.116 ±(99.9%) 4.151 ms/op
[info] Histogram, ms/op:
[info] [ 800.000, 825.000) = 0
[info] [ 825.000, 850.000) = 0
[info] [ 850.000, 875.000) = 0
[info] [ 875.000, 900.000) = 3
[info] [ 900.000, 925.000) = 62
[info] [ 925.000, 950.000) = 209
[info] [ 950.000, 975.000) = 136
[info] [ 975.000, 1000.000) = 59
[info] [1000.000, 1025.000) = 15
[info] [1025.000, 1050.000) = 6
[info] [1050.000, 1075.000) = 3
[info] Percentiles, ms/op:
[info] p(0.0000) = 896.532 ms/op
[info] p(50.0000) = 945.816 ms/op
[info] p(90.0000) = 985.661 ms/op
[info] p(95.0000) = 1002.648 ms/op
[info] p(99.0000) = 1054.867 ms/op
[info] p(99.9000) = 1090.519 ms/op
[info] p(99.9900) = 1090.519 ms/op
[info] p(99.9990) = 1090.519 ms/op
[info] p(99.9999) = 1090.519 ms/op
[info] p(100.0000) = 1090.519 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 303.383 ±(99.9%) 1.892 MB/sec [Average]
[info] (min, avg, max) = (296.837, 303.383, 310.803), stdev = 3.599
[info] CI (99.9%): [301.491, 305.275] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 317189086.400 ±(99.9%) 1025163.244 B/op [Average]
[info] (min, avg, max) = (315741419.636, 317189086.400, 320882423.273), stdev = 1950479.766
[info] CI (99.9%): [316163923.156, 318214249.644] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 303.343 ±(99.9%) 14.464 MB/sec [Average]
[info] (min, avg, max) = (262.424, 303.343, 340.077), stdev = 27.519
[info] CI (99.9%): [288.879, 317.807] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 317127512.436 ±(99.9%) 14913661.565 B/op [Average]
[info] (min, avg, max) = (277824977.455, 317127512.436, 347936581.818), stdev = 28374793.261
[info] CI (99.9%): [302213850.872, 332041174.001] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.448 ±(99.9%) 0.298 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.448, 2.299), stdev = 0.567
[info] CI (99.9%): [0.150, 0.746] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 470730.764 ±(99.9%) 313042.620 B/op [Average]
[info] (min, avg, max) = (≈ 0, 470730.764, 2424792.000), stdev = 595596.165
[info] CI (99.9%): [157688.143, 783773.384] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 250.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.556, 6.000), stdev = 0.503
[info] CI (99.9%): [250.000, 250.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5060.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (51.000, 112.444, 147.000), stdev = 24.973
[info] CI (99.9%): [5060.000, 5060.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:40
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 495 951.116 ± 4.151 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 896.532 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 945.816 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 985.661 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 1002.648 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 1054.867 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 1090.519 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 1090.519 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 1090.519 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 303.383 ± 1.892 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 317189086.400 ± 1025163.244 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 303.343 ± 14.464 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 317127512.436 ± 14913661.565 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 0.448 ± 0.298 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 470730.764 ± 313042.620 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 250.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-f4865e8-SNAPSHOT scalap sample 45 5060.000 ms
[success] Total time: 1017 s, completed Aug 23, 2018 10:17:37 PM
# reduce allocation of Some objects for cached btype lookup
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 952.190 ±(99.9%) 4.718 ms/op
[info] Histogram, ms/op:
[info] [ 800.000, 825.000) = 0
[info] [ 825.000, 850.000) = 0
[info] [ 850.000, 875.000) = 0
[info] [ 875.000, 900.000) = 2
[info] [ 900.000, 925.000) = 101
[info] [ 925.000, 950.000) = 145
[info] [ 950.000, 975.000) = 145
[info] [ 975.000, 1000.000) = 65
[info] [1000.000, 1025.000) = 22
[info] [1025.000, 1050.000) = 9
[info] [1050.000, 1075.000) = 4
[info] Percentiles, ms/op:
[info] p(0.0000) = 894.435 ms/op
[info] p(50.0000) = 948.961 ms/op
[info] p(90.0000) = 989.856 ms/op
[info] p(95.0000) = 1007.052 ms/op
[info] p(99.0000) = 1057.174 ms/op
[info] p(99.9000) = 1088.422 ms/op
[info] p(99.9900) = 1088.422 ms/op
[info] p(99.9990) = 1088.422 ms/op
[info] p(99.9999) = 1088.422 ms/op
[info] p(100.0000) = 1088.422 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 303.899 ±(99.9%) 3.385 MB/sec [Average]
[info] (min, avg, max) = (291.357, 303.899, 316.149), stdev = 6.441
[info] CI (99.9%): [300.513, 307.284] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 317992127.111 ±(99.9%) 999358.621 B/op [Average]
[info] (min, avg, max) = (316309445.091, 317992127.111, 321661677.091), stdev = 1901383.785
[info] CI (99.9%): [316992768.490, 318991485.733] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 304.115 ±(99.9%) 14.975 MB/sec [Average]
[info] (min, avg, max) = (264.323, 304.115, 345.221), stdev = 28.491
[info] CI (99.9%): [289.141, 319.090] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 318210997.560 ±(99.9%) 15225703.883 B/op [Average]
[info] (min, avg, max) = (278253940.364, 318210997.560, 351654260.364), stdev = 28968486.248
[info] CI (99.9%): [302985293.676, 333436701.443] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.391 ±(99.9%) 0.280 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.391, 2.520), stdev = 0.533
[info] CI (99.9%): [0.111, 0.672] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 407114.667 ±(99.9%) 290337.272 B/op [Average]
[info] (min, avg, max) = (≈ 0, 407114.667, 2583405.091), stdev = 552396.876
[info] CI (99.9%): [116777.395, 697451.938] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 250.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.556, 6.000), stdev = 0.503
[info] CI (99.9%): [250.000, 250.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5090.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (52.000, 113.111, 148.000), stdev = 24.484
[info] CI (99.9%): [5090.000, 5090.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:43
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 495 952.190 ± 4.718 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 894.435 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 948.961 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 989.856 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 1007.052 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 1057.174 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 1088.422 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 1088.422 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 1088.422 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 303.899 ± 3.385 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 317992127.111 ± 999358.621 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 304.115 ± 14.975 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 318210997.560 ± 15225703.883 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 0.391 ± 0.280 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 407114.667 ± 290337.272 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 250.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-d103a1e-SNAPSHOT scalap sample 45 5090.000 ms
[success] Total time: 1020 s, completed Aug 23, 2018 10:38:38 PM
# avoid intermediate zipped list building
[info] Result "scala.tools.nsc.HotScalacBenchmark.compile":
[info] N = 495
[info] mean = 950.808 ±(99.9%) 4.359 ms/op
[info] Histogram, ms/op:
[info] [ 900.000, 925.000) = 83
[info] [ 925.000, 950.000) = 192
[info] [ 950.000, 975.000) = 141
[info] [ 975.000, 1000.000) = 55
[info] [1000.000, 1025.000) = 13
[info] [1025.000, 1050.000) = 4
[info] [1050.000, 1075.000) = 3
[info] [1075.000, 1100.000) = 3
[info] [1100.000, 1125.000) = 0
[info] [1125.000, 1150.000) = 1
[info] [1150.000, 1175.000) = 0
[info] Percentiles, ms/op:
[info] p(0.0000) = 901.775 ms/op
[info] p(50.0000) = 945.816 ms/op
[info] p(90.0000) = 983.564 ms/op
[info] p(95.0000) = 999.712 ms/op
[info] p(99.0000) = 1073.826 ms/op
[info] p(99.9000) = 1134.559 ms/op
[info] p(99.9900) = 1134.559 ms/op
[info] p(99.9990) = 1134.559 ms/op
[info] p(99.9999) = 1134.559 ms/op
[info] p(100.0000) = 1134.559 ms/op
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate":
[info] 302.672 ±(99.9%) 1.844 MB/sec [Average]
[info] (min, avg, max) = (292.991, 302.672, 308.737), stdev = 3.508
[info] CI (99.9%): [300.828, 304.516] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.alloc.rate.norm":
[info] 316361041.794 ±(99.9%) 411893.302 B/op [Average]
[info] (min, avg, max) = (315263810.909, 316361041.794, 318109254.545), stdev = 783669.874
[info] CI (99.9%): [315949148.492, 316772935.096] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space":
[info] 302.875 ±(99.9%) 14.495 MB/sec [Average]
[info] (min, avg, max) = (269.275, 302.875, 340.523), stdev = 27.578
[info] CI (99.9%): [288.380, 317.369] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm":
[info] 316627585.293 ±(99.9%) 15356810.718 B/op [Average]
[info] (min, avg, max) = (283353832.727, 316627585.293, 351082309.818), stdev = 29217930.645
[info] CI (99.9%): [301270774.575, 331984396.010] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space":
[info] 0.173 ±(99.9%) 0.251 MB/sec [Average]
[info] (min, avg, max) = (≈ 0, 0.173, 2.362), stdev = 0.478
[info] CI (99.9%): [≈ 0, 0.424] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm":
[info] 179581.964 ±(99.9%) 260815.167 B/op [Average]
[info] (min, avg, max) = (≈ 0, 179581.964, 2448700.364), stdev = 496228.000
[info] CI (99.9%): [≈ 0, 440397.131] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.count":
[info] 247.000 ±(99.9%) 0.001 counts [Sum]
[info] (min, avg, max) = (5.000, 5.489, 6.000), stdev = 0.506
[info] CI (99.9%): [247.000, 247.000] (assumes normal distribution)
[info] Secondary result "scala.tools.nsc.HotScalacBenchmark.compile:·gc.time":
[info] 5469.000 ±(99.9%) 0.001 ms [Sum]
[info] (min, avg, max) = (62.000, 121.533, 148.000), stdev = 20.998
[info] CI (99.9%): [5469.000, 5469.000] (assumes normal distribution)
[info] # Run complete. Total time: 00:16:37
[info] REMEMBER: The numbers below are just data. To gain reusable insights, you need to follow up on
[info] why the numbers are the way they are. Use profilers (see -prof, -lprof), design factorial
[info] experiments, perform baseline and negative tests that provide experimental control, make sure
[info] the benchmarking environment is safe on JVM/OS/HW level, ask for reviews from the domain experts.
[info] Do not assume the numbers tell you what you want them to tell.
[info] Benchmark (corpusPath) (corpusVersion) (extraArgs) (resident) (scalaVersion) (source) Mode Cnt Score Error Units
[info] HotScalacBenchmark.compile ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 495 950.808 ± 4.359 ms/op
[info] HotScalacBenchmark.compile:compile·p0.00 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 901.775 ms/op
[info] HotScalacBenchmark.compile:compile·p0.50 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 945.816 ms/op
[info] HotScalacBenchmark.compile:compile·p0.90 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 983.564 ms/op
[info] HotScalacBenchmark.compile:compile·p0.95 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 999.712 ms/op
[info] HotScalacBenchmark.compile:compile·p0.99 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 1073.826 ms/op
[info] HotScalacBenchmark.compile:compile·p0.999 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 1134.559 ms/op
[info] HotScalacBenchmark.compile:compile·p0.9999 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 1134.559 ms/op
[info] HotScalacBenchmark.compile:compile·p1.00 ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 1134.559 ms/op
[info] HotScalacBenchmark.compile:·gc.alloc.rate ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 302.672 ± 1.844 MB/sec
[info] HotScalacBenchmark.compile:·gc.alloc.rate.norm ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 316361041.794 ± 411893.302 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 302.875 ± 14.495 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Eden_Space.norm ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 316627585.293 ± 15356810.718 B/op
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 0.173 ± 0.251 MB/sec
[info] HotScalacBenchmark.compile:·gc.churn.PS_Survivor_Space.norm ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 179581.964 ± 260815.167 B/op
[info] HotScalacBenchmark.compile:·gc.count ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 247.000 counts
[info] HotScalacBenchmark.compile:·gc.time ../corpus latest false 2.12.6-bin-6fb0646-SNAPSHOT scalap sample 45 5469.000 ms
[success] Total time: 1015 s, completed Aug 23, 2018 11:00:50 PM
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment