Skip to content

Instantly share code, notes, and snippets.

@bollu
Created October 15, 2021 11:17
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bollu/0882e4f9af28d03df712e0f37c5e6979 to your computer and use it in GitHub Desktop.
Save bollu/0882e4f9af28d03df712e0f37c5e6979 to your computer and use it in GitHub Desktop.
Report for stdlib
stdlib [0] ( 2 single benchmarks)
wall-clock mean = 241.(113), deviation = 10.59262%
stdlib [1] ( 2 single benchmarks)
wall-clock mean = 211.(160), deviation = 20.19648%
Equal program blocks
stdlib [0] ⟷ stdlib [1]
wall-clock confidence = 61%, speed up = 12.42%
Report for tests/bench/ interpreted
tests/bench/ interpreted [0] ( 5 single benchmarks)
wall-clock mean = 13.(250), deviation = 8.01884%
tests/bench/ interpreted [1] ( 5 single benchmarks)
wall-clock mean = 12.8(29), deviation = 0.68077%
Equal program blocks
tests/bench/ interpreted [0] ⟷ tests/bench/ interpreted [1]
wall-clock confidence = 45%, speed up = 3.17%
Report for binarytrees
binarytrees [0] ( 10 single benchmarks)
wall-clock mean = 5.(916), deviation = 5.56497%
binarytrees [1] ( 10 single benchmarks)
wall-clock mean = 5.79(8), deviation = 0.35019%
Equal program blocks
binarytrees [0] ⟷ binarytrees [1]
wall-clock confidence = 30%, speed up = 2.00%
Report for deriv
deriv [0] ( 10 single benchmarks)
wall-clock mean = 1.1(51), deviation = 1.88779%
deriv [1] ( 10 single benchmarks)
wall-clock mean = 1.1(40), deviation = 0.97327%
Equal program blocks
deriv [0] ⟷ deriv [1]
wall-clock confidence = 17%, speed up = 1.00%
Report for const_fold
const_fold [0] ( 10 single benchmarks)
wall-clock mean = 2.(736), deviation = 5.51015%
const_fold [1] ( 10 single benchmarks)
wall-clock mean = 2.6(66), deviation = 1.22204%
Equal program blocks
const_fold [0] ⟷ const_fold [1]
wall-clock confidence = 19%, speed up = 2.56%
Report for qsort
qsort [0] ( 10 single benchmarks)
wall-clock mean = 2.8(44), deviation = 2.45203%
qsort [1] ( 10 single benchmarks)
wall-clock mean = 2.81(9), deviation = 0.27401%
Equal program blocks
qsort [0] ⟷ qsort [1]
wall-clock confidence = 28%, speed up = 0.91%
Report for rbmap
rbmap [0] ( 10 single benchmarks)
wall-clock mean = 80(2).(103)m, deviation = 0.47894%
rbmap [1] ( 10 single benchmarks)
wall-clock mean = 8(02).(992)m, deviation = 0.54048%
Equal program blocks
rbmap [0] ⟷ rbmap [1]
wall-clock confidence = 65%, speed up = -0.11%
Report for rbmap_1
rbmap_1 [0] ( 10 single benchmarks)
wall-clock mean = 3.2(57), deviation = 4.43993%
rbmap_1 [1] ( 10 single benchmarks)
wall-clock mean = 3.2(22), deviation = 0.81786%
Equal program blocks
rbmap_1 [0] ⟷ rbmap_1 [1]
wall-clock confidence = 48%, speed up = 1.09%
Report for rbmap_10
rbmap_10 [0] ( 10 single benchmarks)
wall-clock mean = 1.11(7), deviation = 0.29099%
rbmap_10 [1] ( 10 single benchmarks)
wall-clock mean = 1.11(1), deviation = 0.49782%
Unequal program blocks
rbmap_10 [0] ⟷ rbmap_10 [1]
wall-clock confidence = 98%, speed up = 0.50%
Report for unionfind
unionfind [0] ( 10 single benchmarks)
wall-clock mean = 2.(681), deviation = 5.74850%
unionfind [1] ( 10 single benchmarks)
wall-clock mean = 2.6(13), deviation = 3.31926%
Equal program blocks
unionfind [0] ⟷ unionfind [1]
wall-clock confidence = 26%, speed up = 2.53%
Errorneous runs
tests/compiler:
Not a valid benchmarking program output: sum binary sizes:
Traceback (most recent call last):
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 688, in benchmark
res = self._benchmark(block, runs, cpuset, set_id, timeout=timeout)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 717, in _benchmark
res = self.runner.parse_result(exec_res, res, block["parse_output"]
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1007, in parse_result
ret = self.parse_result_impl(exec_res, res)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1535, in parse_result_impl
raise BenchmarkingError("Not a valid benchmarking program output: " + exec_res.stdout)
temci.run.run_driver.BenchmarkingError: Not a valid benchmarking program output: sum binary sizes:
tests/compiler:
Not a valid benchmarking program output: sum binary sizes:
Traceback (most recent call last):
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 688, in benchmark
res = self._benchmark(block, runs, cpuset, set_id, timeout=timeout)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 717, in _benchmark
res = self.runner.parse_result(exec_res, res, block["parse_output"]
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1007, in parse_result
ret = self.parse_result_impl(exec_res, res)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1535, in parse_result_impl
raise BenchmarkingError("Not a valid benchmarking program output: " + exec_res.stdout)
temci.run.run_driver.BenchmarkingError: Not a valid benchmarking program output: sum binary sizes:
Report for single runs
stdlib ( 2 single benchmarks)
.olean serialization mean = 18.(635), deviation = 23.23473%
C code generation mean = 4.(688), deviation = 26.98396%
branch-misses mean = 10.22930G, deviation = 1.39827%
branches mean = 475.24425G, deviation = 0.00773%
compilation mean = 378.(668), deviation = 26.27144%
elaboration mean = 163.(118), deviation = 24.51874%
import mean = 46.(673), deviation = 22.61782%
initialization mean = 20.(391), deviation = 20.88832%
instructions mean = 1.97153T, deviation = 0.01118%
interpretation mean = 1.(087), deviation = 20.48106%
maxrss mean = 317.29800k, deviation = 0.02584%
parsing mean = 10.(721), deviation = 22.90409%
simp mean = 5(38).(335)m, deviation = 2.30061%
task-clock mean = 782.(209), deviation = 24.58304%
typeclass inference mean = 71.(482), deviation = 28.30617%
wall-clock mean = 211.(160), deviation = 20.19648%
stdlib size ( 1 single benchmarks)
bytes .olean mean = 262.096M, deviation = 0.00000%
lines mean = 83.864k, deviation = 0.00000%
lines C mean = 2.779M, deviation = 0.00000%
lines C++ mean = 41.957k, deviation = 0.00000%
libleanshared.so ( 1 single benchmarks)
binary size mean = 49.387M, deviation = 0.00000%
tests/bench/ interpreted ( 5 single benchmarks)
branch-misses mean = 147.46000M, deviation = 0.63065%
branches mean = 22.00047G, deviation = 0.00919%
instructions mean = 105.45730G, deviation = 0.00956%
maxrss mean = 129.79360k, deviation = 0.15362%
task-clock mean = 12.8(29), deviation = 0.67886%
wall-clock mean = 12.8(29), deviation = 0.68077%
binarytrees ( 10 single benchmarks)
branch-misses mean = 61.47248M, deviation = 1.82754%
branches mean = 11.14781G, deviation = 0.00384%
instructions mean = 47.64203G, deviation = 0.00444%
maxrss mean = 137.92080k, deviation = 0.11267%
task-clock mean = 5.79(8), deviation = 0.34979%
wall-clock mean = 5.79(8), deviation = 0.35019%
deriv ( 10 single benchmarks)
branch-misses mean = 11.71006M, deviation = 2.08674%
branches mean = 1.74862G, deviation = 0.00196%
instructions mean = 7.27619G, deviation = 0.00197%
maxrss mean = 441.11600k, deviation = 0.02536%
task-clock mean = 1.1(39), deviation = 0.97388%
wall-clock mean = 1.1(40), deviation = 0.97327%
const_fold ( 10 single benchmarks)
branch-misses mean = 31.66541M, deviation = 2.10424%
branches mean = 2.25972G, deviation = 0.00147%
instructions mean = 10.76928G, deviation = 0.00168%
maxrss mean = 1.82764M, deviation = 0.00559%
task-clock mean = 2.6(66), deviation = 1.22306%
wall-clock mean = 2.6(66), deviation = 1.22204%
qsort ( 10 single benchmarks)
branch-misses mean = 156.37104M, deviation = 1.15164%
branches mean = 4.95749G, deviation = 0.00124%
instructions mean = 22.58314G, deviation = 0.00132%
maxrss mean = 62.64360k, deviation = 0.03274%
task-clock mean = 2.81(8), deviation = 0.27398%
wall-clock mean = 2.81(9), deviation = 0.27401%
rbmap ( 10 single benchmarks)
branch-misses mean = 30.35415M, deviation = 0.19736%
branches mean = 1.31106G, deviation = 0.00205%
instructions mean = 6.08188G, deviation = 0.00203%
maxrss mean = 88.55720k, deviation = 0.10522%
task-clock mean = 8(02).(564)m, deviation = 0.53713%
wall-clock mean = 8(02).(992)m, deviation = 0.54048%
rbmap_1 ( 10 single benchmarks)
branch-misses mean = 43.38408M, deviation = 0.53349%
branches mean = 3.77973G, deviation = 0.00879%
instructions mean = 17.66126G, deviation = 0.00857%
maxrss mean = 2.37034M, deviation = 0.00534%
task-clock mean = 3.2(21), deviation = 0.81767%
wall-clock mean = 3.2(22), deviation = 0.81786%
rbmap_10 ( 10 single benchmarks)
branch-misses mean = 32.59225M, deviation = 0.22870%
branches mean = 1.56731G, deviation = 0.00144%
instructions mean = 7.25715G, deviation = 0.00077%
maxrss mean = 320.71760k, deviation = 0.04387%
task-clock mean = 1.11(1), deviation = 0.49634%
wall-clock mean = 1.11(1), deviation = 0.49782%
unionfind ( 10 single benchmarks)
branch-misses mean = 592.0531k, deviation = 1.14610%
branches mean = 6.18677G, deviation = 0.00231%
instructions mean = 27.28085G, deviation = 0.00251%
maxrss mean = 104.00920k, deviation = 0.12795%
task-clock mean = 2.6(12), deviation = 3.31750%
wall-clock mean = 2.6(13), deviation = 3.31926%
Errorneous runs
tests/compiler:
Not a valid benchmarking program output: sum binary sizes:
Traceback (most recent call last):
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 688, in benchmark
res = self._benchmark(block, runs, cpuset, set_id, timeout=timeout)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 717, in _benchmark
res = self.runner.parse_result(exec_res, res, block["parse_output"]
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1007, in parse_result
ret = self.parse_result_impl(exec_res, res)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1535, in parse_result_impl
raise BenchmarkingError("Not a valid benchmarking program output: " + exec_res.stdout)
temci.run.run_driver.BenchmarkingError: Not a valid benchmarking program output: sum binary sizes:
- attributes:
__description: stdlib
description: stdlib
tags:
- slow
time:
perf_stat:
properties:
- wall-clock
- task-clock
- instructions
- branches
- branch-misses
runner: perf_stat
rusage_properties:
- maxrss
data:
.olean serialization:
- 14.30487
- 22.96425
C code generation:
- 3.423046
- 5.953102
__ov-time:
- 168.60135197639465
- 253.83382987976074
branch-misses:
- 10086264877
- 10372330509
branches:
- 475207510252
- 475280980763
compilation:
- 279.186298
- 478.149221
elaboration:
- 123.12331
- 203.11214
import:
- 36.116553
- 57.22936
initialization:
- 16.1319
- 24.6507
instructions:
- 1971312987301
- 1971753969914
interpretation:
- 0.864056
- 1.309152
maxrss:
- 317380
- 317216
parsing:
- 8.265195
- 13.176137
simp:
- 0.52595
- 0.55072
task-clock:
- 589.91811
- 974.49955
typeclass inference:
- 51.248347
- 91.716107
wall-clock:
- 168.512867689
- 253.80653381
- attributes:
__description: stdlib size
description: stdlib size
tags:
- deterministic
- fast
data:
bytes .olean:
- 262095544
lines:
- 83864
lines C:
- 2779175
lines C++:
- 41957
- attributes:
__description: libleanshared.so
description: libleanshared.so
tags:
- deterministic
- fast
data:
binary size:
- 49386808
- attributes:
description: tests/compiler
tags:
- deterministic
- slow
data: {}
internal_error:
message: "Not a valid benchmarking program output: sum binary sizes: \nTraceback\
\ (most recent call last):\n\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 688, in benchmark\n res = self._benchmark(block, runs, cpuset, set_id,\
\ timeout=timeout)\n\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 717, in _benchmark\n res = self.runner.parse_result(exec_res, res,\
\ block[\"parse_output\"]\n\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 1007, in parse_result\n ret = self.parse_result_impl(exec_res, res)\n\
\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 1535, in parse_result_impl\n raise BenchmarkingError(\"Not a valid\
\ benchmarking program output: \" + exec_res.stdout)\n\ntemci.run.run_driver.BenchmarkingError:\
\ Not a valid benchmarking program output: sum binary sizes: \n"
- attributes:
__description: tests/bench/ interpreted
description: tests/bench/ interpreted
tags:
- slow
data:
__ov-time:
- 12.968934059143066
- 12.795540809631348
- 12.715871572494507
- 12.908612966537476
- 12.884806394577026
branch-misses:
- 148807865
- 148272626
- 146380583
- 146698676
- 147140259
branches:
- 22000317065
- 22004403037
- 21998835209
- 21999369229
- 21999430681
instructions:
- 105456538695
- 105476913774
- 105449289101
- 105451695241
- 105452056451
maxrss:
- 129668
- 129916
- 129460
- 129960
- 129964
task-clock:
- 12.93294
- 12.757940000000001
- 12.69844
- 12.89015
- 12.86485
wall-clock:
- 12.933908243
- 12.758449824
- 12.698658286
- 12.890935627
- 12.865521085
- attributes:
__description: binarytrees
description: binarytrees
tags:
- fast
- suite
data:
__ov-time:
- 5.7995641231536865
- 5.821425437927246
- 5.8118510246276855
- 5.818291187286377
- 5.7881920337677
- 5.813392400741577
- 5.808026075363159
- 5.8481690883636475
- 5.856918573379517
- 5.802680492401123
branch-misses:
- 60164884
- 60371260
- 62724901
- 64135810
- 60802101
- 61275303
- 61150584
- 61024842
- 61194098
- 61881053
branches:
- 11147623170
- 11147505209
- 11147471859
- 11148630942
- 11147482220
- 11147561282
- 11147540449
- 11147803166
- 11147843927
- 11148629461
instructions:
- 47641132361
- 47640559853
- 47640418412
- 47646085027
- 47640413649
- 47640814807
- 47640725805
- 47641948932
- 47642065967
- 47646142990
maxrss:
- 138068
- 137728
- 138068
- 137708
- 137816
- 138136
- 137920
- 137968
- 137732
- 138064
task-clock:
- 5.77629
- 5.79954
- 5.79421
- 5.800020000000001
- 5.7695799999999995
- 5.79392
- 5.78975
- 5.828930000000001
- 5.8381099999999995
- 5.78518
wall-clock:
- 5.776604857
- 5.799684222
- 5.794378784
- 5.800307004
- 5.769718332
- 5.794399318
- 5.79002094
- 5.829148188
- 5.838486992
- 5.785440882
- attributes:
__description: deriv
description: deriv
tags:
- fast
- suite
data:
__ov-time:
- 1.1679306030273438
- 1.1540229320526123
- 1.153876543045044
- 1.159144639968872
- 1.1575148105621338
- 1.1490318775177002
- 1.1505990028381348
- 1.1457087993621826
- 1.153073787689209
- 1.189427137374878
branch-misses:
- 11342099
- 11529450
- 12060958
- 11786157
- 12059715
- 11836979
- 11480972
- 11775340
- 11822760
- 11406146
branches:
- 1748690302
- 1748604743
- 1748623692
- 1748622794
- 1748574380
- 1748598998
- 1748622844
- 1748558893
- 1748616045
- 1748640762
instructions:
- 7276595752
- 7276168436
- 7276178724
- 7276139737
- 7276096969
- 7276155071
- 7276192575
- 7276023636
- 7276161967
- 7276188935
maxrss:
- 441164
- 441152
- 441116
- 441024
- 441060
- 441032
- 441336
- 441060
- 441268
- 440948
task-clock:
- 1.1492200000000001
- 1.13443
- 1.13628
- 1.1408399999999999
- 1.13853
- 1.13056
- 1.13283
- 1.12769
- 1.13439
- 1.1679300000000001
wall-clock:
- 1.14959923
- 1.134851775
- 1.136736435
- 1.141205856
- 1.138997111
- 1.131004903
- 1.133320445
- 1.128114699
- 1.134780177
- 1.168381572
- attributes:
__description: const_fold
description: const_fold
tags:
- fast
- suite
data:
__ov-time:
- 2.7330141067504883
- 2.656216621398926
- 2.6684484481811523
- 2.716933012008667
- 2.6674447059631348
- 2.6601035594940186
- 2.7298548221588135
- 2.649200916290283
- 2.650892734527588
- 2.714048147201538
branch-misses:
- 32275182
- 31565293
- 30633771
- 32244078
- 31929988
- 31863494
- 32816952
- 30910327
- 31549521
- 30865450
branches:
- 2259805781
- 2259705943
- 2259726171
- 2259715125
- 2259719141
- 2259707184
- 2259717154
- 2259684756
- 2259675053
- 2259709481
instructions:
- 10769767027
- 10769242968
- 10769300406
- 10769244832
- 10769316354
- 10769225811
- 10769287916
- 10769100952
- 10769057698
- 10769276713
maxrss:
- 1827812
- 1827584
- 1827856
- 1827612
- 1827568
- 1827552
- 1827628
- 1827624
- 1827536
- 1827628
task-clock:
- 2.71415
- 2.63811
- 2.65012
- 2.69815
- 2.64837
- 2.64216
- 2.71157
- 2.62954
- 2.63207
- 2.69451
wall-clock:
- 2.714540804
- 2.638453398
- 2.650691268
- 2.698573455
- 2.648823723
- 2.642727793
- 2.71197092
- 2.629970388
- 2.632476119
- 2.694905103
- attributes:
__description: qsort
description: qsort
tags:
- fast
- suite
data:
__ov-time:
- 2.840747117996216
- 2.8545897006988525
- 2.829132318496704
- 2.8440680503845215
- 2.8366096019744873
- 2.8416123390197754
- 2.8256163597106934
- 2.8430728912353516
- 2.827208995819092
- 2.8338303565979004
branch-misses:
- 156903428
- 159045914
- 152538575
- 158585345
- 157296923
- 156488726
- 154939617
- 157075797
- 155128876
- 155707159
branches:
- 4957540348
- 4957434512
- 4957656437
- 4957465677
- 4957470722
- 4957466971
- 4957457268
- 4957476776
- 4957467984
- 4957453866
instructions:
- 22583496906
- 22582863176
- 22583905764
- 22583052562
- 22583056212
- 22583038100
- 22582974670
- 22583063911
- 22583040649
- 22582957989
maxrss:
- 62592
- 62624
- 62640
- 62644
- 62648
- 62648
- 62656
- 62660
- 62660
- 62664
task-clock:
- 2.8206100000000003
- 2.83388
- 2.81109
- 2.82429
- 2.8183800000000003
- 2.8185700000000002
- 2.8072600000000003
- 2.8236399999999997
- 2.8081199999999997
- 2.81579
wall-clock:
- 2.820968955
- 2.834259964
- 2.811468916
- 2.824631167
- 2.818759103
- 2.818964465
- 2.807642008
- 2.824035895
- 2.808462936
- 2.816176173
- attributes:
__description: rbmap
description: rbmap
tags:
- fast
- suite
data:
__ov-time:
- 0.8191282749176025
- 0.8185625076293945
- 0.8234856128692627
- 0.8249273300170898
- 0.8220131397247314
- 0.8177063465118408
- 0.8235738277435303
- 0.8149981498718262
- 0.8192043304443359
- 0.8325481414794922
branch-misses:
- 30373100
- 30350473
- 30302414
- 30282054
- 30465716
- 30319790
- 30458477
- 30300279
- 30349409
- 30339829
branches:
- 1311117767
- 1311048903
- 1311096969
- 1311050798
- 1311046880
- 1311037096
- 1311063298
- 1311056899
- 1311033673
- 1311093450
instructions:
- 6082155857
- 6081784703
- 6082027465
- 6081840459
- 6081800190
- 6081788294
- 6081883154
- 6081814798
- 6081759451
- 6081983668
maxrss:
- 88588
- 88492
- 88568
- 88632
- 88572
- 88420
- 88544
- 88604
- 88412
- 88740
task-clock:
- 0.7998500000000001
- 0.80011
- 0.8055800000000001
- 0.80659
- 0.80037
- 0.80001
- 0.80355
- 0.7975
- 0.79952
- 0.81256
wall-clock:
- 0.800288645
- 0.800542049
- 0.805994573
- 0.807007098
- 0.800781537
- 0.800403165
- 0.803921178
- 0.797909957
- 0.799958339
- 0.813112675
- attributes:
__description: rbmap_1
description: rbmap_1
tags:
- fast
- suite
data:
__ov-time:
- 3.3165836334228516
- 3.2524092197418213
- 3.2552976608276367
- 3.271925926208496
- 3.30132794380188
- 3.286837100982666
- 3.244563579559326
- 3.2090322971343994
- 3.2963192462921143
- 3.2284116744995117
branch-misses:
- 43507071
- 43534654
- 43051345
- 43089739
- 43628343
- 43207030
- 43252017
- 43622726
- 43230024
- 43717840
branches:
- 3779510415
- 3779384523
- 3779971306
- 3779380162
- 3779682521
- 3779846541
- 3779965349
- 3780471038
- 3779677169
- 3779364686
instructions:
- 17661202287
- 17660179999
- 17661317383
- 17660181381
- 17660782092
- 17661161739
- 17661332444
- 17665591878
- 17660777454
- 17660096227
maxrss:
- 2370560
- 2370360
- 2370360
- 2370228
- 2370136
- 2370276
- 2370296
- 2370296
- 2370560
- 2370324
task-clock:
- 3.27452
- 3.1933000000000002
- 3.23719
- 3.21394
- 3.23039
- 3.23471
- 3.22512
- 3.1909699999999996
- 3.2312600000000002
- 3.17985
wall-clock:
- 3.274977626
- 3.193811536
- 3.237599552
- 3.214418086
- 3.230783272
- 3.235189743
- 3.225428813
- 3.191402959
- 3.231718489
- 3.180184132
- attributes:
__description: rbmap_10
description: rbmap_10
tags:
- fast
- suite
data:
__ov-time:
- 1.1398546695709229
- 1.1260712146759033
- 1.1379516124725342
- 1.1292304992675781
- 1.1284089088439941
- 1.1283695697784424
- 1.1222646236419678
- 1.1274676322937012
- 1.1330633163452148
- 1.1241025924682617
branch-misses:
- 32556113
- 32547957
- 32506197
- 32637120
- 32703103
- 32635968
- 32457322
- 32664079
- 32656613
- 32558067
branches:
- 1567317834
- 1567303808
- 1567321886
- 1567289004
- 1567299235
- 1567332097
- 1567309830
- 1567302645
- 1567364173
- 1567281079
instructions:
- 7257149657
- 7257120325
- 7257180144
- 7257102501
- 7257152331
- 7257221821
- 7257135069
- 7257178800
- 7257255625
- 7257051404
maxrss:
- 320948
- 320684
- 320612
- 320692
- 320844
- 320684
- 320820
- 320852
- 320584
- 320456
task-clock:
- 1.12055
- 1.10595
- 1.11978
- 1.10846
- 1.10694
- 1.1103800000000001
- 1.10375
- 1.10947
- 1.11452
- 1.10643
wall-clock:
- 1.120950429
- 1.106318534
- 1.120350594
- 1.108910723
- 1.107367913
- 1.110780811
- 1.104169637
- 1.109897114
- 1.114914331
- 1.106901388
- attributes:
__description: unionfind
description: unionfind
tags:
- fast
- suite
data:
__ov-time:
- 2.869389295578003
- 2.579188346862793
- 2.595147132873535
- 2.587369441986084
- 2.698946952819824
- 2.591240882873535
- 2.62941837310791
- 2.5924274921417236
- 2.5892534255981445
- 2.5839977264404297
branch-misses:
- 598656
- 593886
- 585648
- 590137
- 599783
- 589138
- 585286
- 582710
- 590400
- 604887
branches:
- 6187126248
- 6186688911
- 6186712758
- 6186681979
- 6186970229
- 6186698694
- 6186714493
- 6186705307
- 6186727022
- 6186695627
instructions:
- 27282567990
- 27280446406
- 27280541708
- 27280416620
- 27281762323
- 27280473880
- 27280618226
- 27280533101
- 27280642347
- 27280497754
maxrss:
- 104240
- 104192
- 103888
- 103948
- 103948
- 103820
- 103876
- 104092
- 104096
- 103992
task-clock:
- 2.85097
- 2.5575900000000003
- 2.57612
- 2.56881
- 2.68027
- 2.5695300000000003
- 2.61065
- 2.57253
- 2.57084
- 2.56546
wall-clock:
- 2.851454792
- 2.557913423
- 2.576489877
- 2.569138319
- 2.680825229
- 2.569836719
- 2.61099231
- 2.572847707
- 2.571197969
- 2.565808309
Report for single runs
stdlib ( 2 single benchmarks)
.olean serialization mean = 21.(308), deviation = 15.58845%
C code generation mean = 5.(510), deviation = 18.11712%
branch-misses mean = 10.32860G, deviation = 0.78232%
branches mean = 475.26658G, deviation = 0.00353%
compilation mean = 433.(629), deviation = 9.80501%
elaboration mean = 188.(430), deviation = 12.09535%
import mean = 55.(564), deviation = 6.47222%
initialization mean = 24.(370), deviation = 17.10560%
instructions mean = 1.97164T, deviation = 0.00551%
interpretation mean = 1.(334), deviation = 18.70293%
maxrss mean = 317.38400k, deviation = 0.05167%
parsing mean = 12.(836), deviation = 8.70760%
simp mean = (574).(175)m, deviation = 7.58392%
task-clock mean = 912.(545), deviation = 11.79471%
typeclass inference mean = 82.(122), deviation = 9.13051%
wall-clock mean = 241.(113), deviation = 10.59262%
stdlib size ( 1 single benchmarks)
bytes .olean mean = 262.101M, deviation = 0.00000%
lines mean = 83.864k, deviation = 0.00000%
lines C mean = 2.779M, deviation = 0.00000%
lines C++ mean = 41.957k, deviation = 0.00000%
libleanshared.so ( 1 single benchmarks)
binary size mean = 49.387M, deviation = 0.00000%
tests/bench/ interpreted ( 5 single benchmarks)
branch-misses mean = 146.20509M, deviation = 0.63703%
branches mean = 21.99914G, deviation = 0.00155%
instructions mean = 105.45070G, deviation = 0.00164%
maxrss mean = 129.83600k, deviation = 0.14428%
task-clock mean = 13.(250), deviation = 8.01646%
wall-clock mean = 13.(250), deviation = 8.01884%
binarytrees ( 10 single benchmarks)
branch-misses mean = 61.50464M, deviation = 1.84685%
branches mean = 11.14805G, deviation = 0.00622%
instructions mean = 47.64326G, deviation = 0.00723%
maxrss mean = 137.93000k, deviation = 0.08703%
task-clock mean = 5.(916), deviation = 5.56476%
wall-clock mean = 5.(916), deviation = 5.56497%
deriv ( 10 single benchmarks)
branch-misses mean = 11.89409M, deviation = 2.31366%
branches mean = 1.74863G, deviation = 0.00247%
instructions mean = 7.27609G, deviation = 0.00211%
maxrss mean = 441.04400k, deviation = 0.02933%
task-clock mean = 1.1(51), deviation = 1.88620%
wall-clock mean = 1.1(51), deviation = 1.88779%
const_fold ( 10 single benchmarks)
branch-misses mean = 31.62363M, deviation = 2.02436%
branches mean = 2.25969G, deviation = 0.00292%
instructions mean = 10.76898G, deviation = 0.00317%
maxrss mean = 1.82775M, deviation = 0.00644%
task-clock mean = 2.(736), deviation = 5.50963%
wall-clock mean = 2.(736), deviation = 5.51015%
qsort ( 10 single benchmarks)
branch-misses mean = 156.25014M, deviation = 0.85254%
branches mean = 4.95749G, deviation = 0.00095%
instructions mean = 22.58317G, deviation = 0.00112%
maxrss mean = 62.60480k, deviation = 0.03646%
task-clock mean = 2.8(44), deviation = 2.44721%
wall-clock mean = 2.8(44), deviation = 2.45203%
rbmap ( 10 single benchmarks)
branch-misses mean = 30.33410M, deviation = 0.22858%
branches mean = 1.31108G, deviation = 0.00219%
instructions mean = 6.08193G, deviation = 0.00243%
maxrss mean = 88.57200k, deviation = 0.11877%
task-clock mean = 80(1).(664)m, deviation = 0.47828%
wall-clock mean = 80(2).(103)m, deviation = 0.47894%
rbmap_1 ( 10 single benchmarks)
branch-misses mean = 43.32905M, deviation = 0.41966%
branches mean = 3.77964G, deviation = 0.01017%
instructions mean = 17.66082G, deviation = 0.00929%
maxrss mean = 2.37036M, deviation = 0.00403%
task-clock mean = 3.2(57), deviation = 4.44108%
wall-clock mean = 3.2(57), deviation = 4.43993%
rbmap_10 ( 10 single benchmarks)
branch-misses mean = 32.61664M, deviation = 0.21716%
branches mean = 1.56729G, deviation = 0.00216%
instructions mean = 7.25697G, deviation = 0.00106%
maxrss mean = 320.68720k, deviation = 0.02966%
task-clock mean = 1.11(6), deviation = 0.29286%
wall-clock mean = 1.11(7), deviation = 0.29099%
unionfind ( 10 single benchmarks)
branch-misses mean = 595.7799k, deviation = 1.37834%
branches mean = 6.18695G, deviation = 0.00623%
instructions mean = 27.28175G, deviation = 0.00710%
maxrss mean = 104.04680k, deviation = 0.07194%
task-clock mean = 2.(680), deviation = 5.74857%
wall-clock mean = 2.(681), deviation = 5.74850%
Errorneous runs
tests/compiler:
Not a valid benchmarking program output: sum binary sizes:
Traceback (most recent call last):
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 688, in benchmark
res = self._benchmark(block, runs, cpuset, set_id, timeout=timeout)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 717, in _benchmark
res = self.runner.parse_result(exec_res, res, block["parse_output"]
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1007, in parse_result
ret = self.parse_result_impl(exec_res, res)
File "/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py", line 1535, in parse_result_impl
raise BenchmarkingError("Not a valid benchmarking program output: " + exec_res.stdout)
temci.run.run_driver.BenchmarkingError: Not a valid benchmarking program output: sum binary sizes:
- attributes:
__description: stdlib
description: stdlib
tags:
- slow
time:
perf_stat:
properties:
- wall-clock
- task-clock
- instructions
- branches
- branch-misses
runner: perf_stat
rusage_properties:
- maxrss
data:
.olean serialization:
- 17.9861
- 24.62916
C code generation:
- 4.512076
- 6.508728
__ov-time:
- 215.6731173992157
- 266.6829400062561
branch-misses:
- 10247798027
- 10409404410
branches:
- 475249800904
- 475283350095
compilation:
- 391.112042
- 476.14683
elaboration:
- 165.6386
- 211.22112
import:
- 51.967311
- 59.159695
initialization:
- 20.2012
- 28.5384
instructions:
- 1971535530099
- 1971752940246
interpretation:
- 1.084281
- 1.583173
maxrss:
- 317548
- 317220
parsing:
- 11.718375
- 13.953806
simp:
- 0.53063
- 0.61772
task-clock:
- 804.9130200000001
- 1020.17715
typeclass inference:
- 74.623909
- 89.620236
wall-clock:
- 215.57321728
- 266.653667326
- attributes:
__description: stdlib size
description: stdlib size
tags:
- deterministic
- fast
data:
bytes .olean:
- 262100848
lines:
- 83864
lines C:
- 2779222
lines C++:
- 41957
- attributes:
__description: libleanshared.so
description: libleanshared.so
tags:
- deterministic
- fast
data:
binary size:
- 49386808
- attributes:
description: tests/compiler
tags:
- deterministic
- slow
data: {}
internal_error:
message: "Not a valid benchmarking program output: sum binary sizes: \nTraceback\
\ (most recent call last):\n\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 688, in benchmark\n res = self._benchmark(block, runs, cpuset, set_id,\
\ timeout=timeout)\n\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 717, in _benchmark\n res = self.runner.parse_result(exec_res, res,\
\ block[\"parse_output\"]\n\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 1007, in parse_result\n ret = self.parse_result_impl(exec_res, res)\n\
\n File \"/home/bollu/.local/lib/python3.9/site-packages/temci/run/run_driver.py\"\
, line 1535, in parse_result_impl\n raise BenchmarkingError(\"Not a valid\
\ benchmarking program output: \" + exec_res.stdout)\n\ntemci.run.run_driver.BenchmarkingError:\
\ Not a valid benchmarking program output: sum binary sizes: \n"
- attributes:
__description: tests/bench/ interpreted
description: tests/bench/ interpreted
tags:
- slow
data:
__ov-time:
- 15.40156888961792
- 12.6881263256073
- 12.708075046539307
- 12.740128993988037
- 12.827307939529419
branch-misses:
- 147072587
- 146462533
- 146984365
- 145993961
- 144511987
branches:
- 21999745782
- 21998759026
- 21998883493
- 21999097232
- 21999191201
instructions:
- 105453838567
- 105448820076
- 105449438020
- 105450534030
- 105450889831
maxrss:
- 129904
- 129904
- 129908
- 129996
- 129468
task-clock:
- 15.371450000000001
- 12.6571
- 12.689530000000001
- 12.72133
- 12.80834
wall-clock:
- 15.372770803
- 12.657356883
- 12.690149489
- 12.721720338
- 12.808733377
- attributes:
__description: binarytrees
description: binarytrees
tags:
- fast
- suite
data:
__ov-time:
- 6.923649311065674
- 5.810955047607422
- 5.860105752944946
- 5.825525283813477
- 5.835500001907349
- 5.783747673034668
- 5.805765390396118
- 5.850724935531616
- 5.807452201843262
- 5.858917713165283
branch-misses:
- 60713749
- 61319948
- 63636806
- 61929474
- 63592545
- 61319217
- 60609907
- 60829619
- 60416235
- 60678918
branches:
- 11147966926
- 11147566268
- 11149726522
- 11148590986
- 11147543414
- 11147535216
- 11148636557
- 11147569141
- 11147483493
- 11147849087
instructions:
- 47642918469
- 47640874388
- 47651596158
- 47645992926
- 47640725070
- 47640676001
- 47646187695
- 47640935766
- 47640479491
- 47642188531
maxrss:
- 137900
- 137812
- 137888
- 137976
- 137732
- 137984
- 137900
- 138120
- 137856
- 138132
task-clock:
- 6.901
- 5.78847
- 5.84159
- 5.80612
- 5.81446
- 5.76581
- 5.78801
- 5.82644
- 5.78682
- 5.84042
wall-clock:
- 6.901373617
- 5.788882339
- 5.8418264
- 5.806284981
- 5.814760973
- 5.766019825
- 5.788195353
- 5.827182852
- 5.786944477
- 5.840629982
- attributes:
__description: deriv
description: deriv
tags:
- fast
- suite
data:
__ov-time:
- 1.1604907512664795
- 1.154345989227295
- 1.1583564281463623
- 1.1619560718536377
- 1.2335128784179688
- 1.175455093383789
- 1.1655099391937256
- 1.1758606433868408
- 1.160461187362671
- 1.1624491214752197
branch-misses:
- 11826236
- 11714763
- 12599663
- 11775820
- 11860496
- 11528199
- 11918325
- 11689209
- 11979845
- 12048303
branches:
- 1748694492
- 1748559956
- 1748618274
- 1748633961
- 1748608690
- 1748668724
- 1748562067
- 1748681336
- 1748617298
- 1748614923
instructions:
- 7276437523
- 7275870720
- 7276076975
- 7276069517
- 7276094006
- 7276172834
- 7275887450
- 7276214939
- 7276060129
- 7276035742
maxrss:
- 440980
- 440824
- 441272
- 440892
- 441152
- 440984
- 440988
- 441056
- 441132
- 441160
task-clock:
- 1.14125
- 1.1353900000000001
- 1.13955
- 1.14169
- 1.21386
- 1.15645
- 1.14768
- 1.1472
- 1.14181
- 1.14297
wall-clock:
- 1.141641685
- 1.135783551
- 1.13994451
- 1.142109477
- 1.214360119
- 1.156947865
- 1.148058321
- 1.147760125
- 1.142215936
- 1.143446458
- attributes:
__description: const_fold
description: const_fold
tags:
- fast
- suite
data:
__ov-time:
- 3.1991379261016846
- 2.6625559329986572
- 2.730678081512451
- 2.663818836212158
- 2.669412612915039
- 2.7436230182647705
- 2.7425363063812256
- 2.740844249725342
- 2.6730775833129883
- 2.7298381328582764
branch-misses:
- 31611064
- 31124785
- 31809483
- 30857934
- 31543514
- 33279037
- 31886269
- 31691805
- 31411152
- 31021265
branches:
- 2259888325
- 2259671982
- 2259678867
- 2259658792
- 2259654416
- 2259675857
- 2259667112
- 2259670489
- 2259673593
- 2259679240
instructions:
- 10770002138
- 10768861339
- 10768933089
- 10768817933
- 10768807041
- 10768887975
- 10768842607
- 10768882227
- 10768882576
- 10768908267
maxrss:
- 1827920
- 1827656
- 1827776
- 1827584
- 1827788
- 1827924
- 1827756
- 1827564
- 1827812
- 1827700
task-clock:
- 3.17687
- 2.64292
- 2.7124200000000003
- 2.64444
- 2.64961
- 2.7237
- 2.72351
- 2.71868
- 2.6552399999999996
- 2.71067
wall-clock:
- 3.177430108
- 2.643364089
- 2.712818877
- 2.644880358
- 2.650033523
- 2.724066183
- 2.723887103
- 2.719073582
- 2.655682993
- 2.711104031
- attributes:
__description: qsort
description: qsort
tags:
- fast
- suite
data:
__ov-time:
- 3.107306480407715
- 2.8352811336517334
- 2.8462274074554443
- 2.831214666366577
- 2.841698408126831
- 2.853377342224121
- 2.8348541259765625
- 2.8318819999694824
- 2.838385581970215
- 2.846527099609375
branch-misses:
- 155822531
- 155167353
- 157145419
- 155435892
- 156690705
- 159169844
- 157267488
- 155594755
- 154075798
- 156131619
branches:
- 4957613527
- 4957466280
- 4957471585
- 4957472716
- 4957519059
- 4957448157
- 4957453250
- 4957464783
- 4957518352
- 4957507033
instructions:
- 22583847902
- 22583028456
- 22583076092
- 22583061281
- 22583268323
- 22582934336
- 22582967354
- 22583022451
- 22583251200
- 22583209362
maxrss:
- 62548
- 62576
- 62608
- 62608
- 62612
- 62616
- 62616
- 62620
- 62620
- 62624
task-clock:
- 3.0518899999999998
- 2.81625
- 2.8261100000000003
- 2.8121300000000002
- 2.82335
- 2.83367
- 2.81706
- 2.8126700000000002
- 2.81961
- 2.8272600000000003
wall-clock:
- 3.052771397
- 2.816620494
- 2.826468532
- 2.812510943
- 2.823773804
- 2.833984308
- 2.817341637
- 2.813002591
- 2.820139518
- 2.827772152
- attributes:
__description: rbmap
description: rbmap
tags:
- fast
- suite
data:
__ov-time:
- 0.8183808326721191
- 0.8251152038574219
- 0.8214089870452881
- 0.8192563056945801
- 0.8238387107849121
- 0.8158133029937744
- 0.8170785903930664
- 0.8213677406311035
- 0.8275322914123535
- 0.8215062618255615
branch-misses:
- 30343140
- 30439302
- 30437638
- 30253050
- 30330737
- 30263768
- 30225877
- 30315421
- 30383087
- 30348990
branches:
- 1311135023
- 1311084962
- 1311058835
- 1311053493
- 1311116924
- 1311040947
- 1311060446
- 1311058608
- 1311095817
- 1311066659
instructions:
- 6082292897
- 6081915341
- 6081836948
- 6081840609
- 6082076430
- 6081762340
- 6081846235
- 6081844242
- 6082003861
- 6081902945
maxrss:
- 88728
- 88628
- 88560
- 88436
- 88536
- 88572
- 88404
- 88540
- 88560
- 88756
task-clock:
- 0.79835
- 0.80664
- 0.8001699999999999
- 0.80087
- 0.80606
- 0.79687
- 0.7981699999999999
- 0.80091
- 0.80874
- 0.79986
wall-clock:
- 0.798847365
- 0.807098465
- 0.80059205
- 0.801232429
- 0.806444133
- 0.797289057
- 0.798566805
- 0.801360567
- 0.809236787
- 0.800365182
- attributes:
__description: rbmap_1
description: rbmap_1
tags:
- fast
- suite
data:
__ov-time:
- 3.7045443058013916
- 3.257155179977417
- 3.2578086853027344
- 3.227921962738037
- 3.310896635055542
- 3.2273106575012207
- 3.242436647415161
- 3.277637004852295
- 3.2163748741149902
- 3.2611324787139893
branch-misses:
- 43413815
- 43110030
- 43565795
- 43454117
- 43038363
- 43452413
- 43216061
- 43371783
- 43545523
- 43122585
branches:
- 3780168840
- 3779363230
- 3779362396
- 3779346932
- 3779352012
- 3779625085
- 3780433738
- 3779797535
- 3779786691
- 3779195050
instructions:
- 17662217169
- 17659917864
- 17659907404
- 17659832658
- 17659849649
- 17660290613
- 17665225519
- 17660745264
- 17660688334
- 17659529440
maxrss:
- 2370328
- 2370560
- 2370340
- 2370332
- 2370408
- 2370324
- 2370164
- 2370396
- 2370436
- 2370352
task-clock:
- 3.68424
- 3.19079
- 3.1966799999999997
- 3.1939
- 3.25085
- 3.2083600000000003
- 3.2085700000000004
- 3.25887
- 3.18279
- 3.1908499999999997
wall-clock:
- 3.684703338
- 3.191482556
- 3.197280227
- 3.19435741
- 3.251325769
- 3.20867545
- 3.209109269
- 3.259211191
- 3.183257207
- 3.191355229
- attributes:
__description: rbmap_10
description: rbmap_10
tags:
- fast
- suite
data:
__ov-time:
- 1.1313996315002441
- 1.1349141597747803
- 1.1343719959259033
- 1.1426966190338135
- 1.1375253200531006
- 1.140077829360962
- 1.1327767372131348
- 1.1384220123291016
- 1.1303069591522217
- 1.1316487789154053
branch-misses:
- 32657005
- 32682361
- 32670177
- 32692430
- 32508114
- 32513539
- 32675880
- 32540744
- 32568891
- 32657291
branches:
- 1567303388
- 1567264859
- 1567248573
- 1567254010
- 1567291417
- 1567324888
- 1567260291
- 1567333308
- 1567334201
- 1567334791
instructions:
- 7256940489
- 7256933515
- 7256862490
- 7256836614
- 7256983800
- 7257010474
- 7256925180
- 7257050560
- 7257069972
- 7257057044
maxrss:
- 320812
- 320736
- 320832
- 320624
- 320576
- 320620
- 320692
- 320552
- 320788
- 320640
task-clock:
- 1.1131099999999998
- 1.11574
- 1.11632
- 1.1216700000000002
- 1.1171900000000001
- 1.12172
- 1.11323
- 1.11771
- 1.11216
- 1.1133199999999999
wall-clock:
- 1.113508754
- 1.116148394
- 1.116697669
- 1.1220985
- 1.117621771
- 1.122059838
- 1.113704184
- 1.118198344
- 1.112552857
- 1.113875417
- attributes:
__description: unionfind
description: unionfind
tags:
- fast
- suite
data:
__ov-time:
- 3.127485990524292
- 2.6848268508911133
- 2.7648119926452637
- 2.585164785385132
- 2.6799890995025635
- 2.5854060649871826
- 2.7034568786621094
- 2.5845744609832764
- 2.601658821105957
- 2.6692919731140137
branch-misses:
- 604549
- 609205
- 590475
- 599981
- 602350
- 586665
- 601226
- 590892
- 588367
- 584089
branches:
- 6188053429
- 6186955623
- 6187009319
- 6186702259
- 6186957978
- 6186706677
- 6186964431
- 6186725876
- 6186727720
- 6186740598
instructions:
- 27287319643
- 27281693372
- 27281950407
- 27280534065
- 27281713309
- 27280548429
- 27281714559
- 27280634691
- 27280657461
- 27280712345
maxrss:
- 103920
- 104036
- 104008
- 104036
- 103992
- 104012
- 104084
- 104036
- 104196
- 104148
task-clock:
- 3.10816
- 2.66636
- 2.74704
- 2.56657
- 2.66121
- 2.56653
- 2.68486
- 2.56646
- 2.58236
- 2.6513400000000003
wall-clock:
- 3.108650346
- 2.666869427
- 2.747716934
- 2.566965953
- 2.661698105
- 2.566975704
- 2.685423795
- 2.566919651
- 2.58279514
- 2.651739017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment