Skip to content

Instantly share code, notes, and snippets.

@AndyAyersMS

AndyAyersMS/8QueensRawData.csv

Last active April 21, 2016 23:46
Show Gist options
  • Save AndyAyersMS/e0c343d9ee9106a35be4 to your computer and use it in GitHub Desktop.
Save AndyAyersMS/e0c343d9ee9106a35be4 to your computer and use it in GitHub Desktop.
Download ZIP
Script for running xunit-perf tests on CoreCLR (windows)
Raw
8QueensRawData.csv
Iteration Time Instructions
1 245.29947067435137 2450300000
2 242.80866366034343 2450400000
3 252.36461047958414 2450500000
4 252.65525905979166 2447100000
5 253.49454765656992 2444600000
6 244.2105881924831 2444400000
7 247.37319930841477 2444500000
8 245.65496701161919 2447900000
9 256.54519142218123 2444500000
10 243.06292285169638 2444400000
1 243.49073143604664 2450100000
2 242.994342849625 2450200000
3 244.335618400341 2450300000
4 246.61648663249798 2447100000
5 243.15949469134807 2444400000
6 241.09089789417385 2444300000
7 245.67036252228854 2444300000
8 241.88913179583506 2447000000
9 242.96728407329783 2444400000
10 245.3792474114548 2445000000
1 245.12405515884905 2450400000
2 246.18121355812639 2450400000
3 245.47815190423808 2450400000
4 240.53106114257093 2447000000
5 239.0176357907369 2444300000
6 239.36240192359924 2444300000
7 237.90682636943029 2444300000
8 245.65916578725682 2446800000
9 243.13103632314142 2444400000
10 239.93203581835542 2444300000
1 243.54578204995425 2450400000
2 242.73915059701915 2450300000
3 242.24649425560847 2450300000
4 240.00061582042667 2447100000
5 237.19256798051015 2444400000
6 240.59870808338951 2444700000
7 237.74167452770735 2444300000
8 238.91173333855841 2446800000
9 238.06684637426406 2444400000
10 238.6966627198176 2444300000
1 242.50261956946679 2450200000
2 241.09836238419553 2450200000
3 242.28101752195721 2450200000
4 241.33582647300045 2446900000
5 239.31574886096564 2444300000
6 239.60593091054625 2444300000
7 240.75546237383878 2444300000
8 239.46970396765619 2446800000
9 237.82938228545891 2444300000
10 238.96725048309236 2444300000
1 243.12823713938349 2450200000
2 244.122880434732 2450300000
3 245.37924741145503 2450200000
4 238.80443129450123 2446900000
5 238.3896855676885 2444300000
6 238.77970517130552 2444300000
7 241.33162769736327 2444500000
8 240.615503185938 2446900000
9 237.90915902256211 2444300000
10 238.42420883403747 2444300000
1 247.08908215697636 2450400000
2 247.03216542056339 2450500000
3 250.78493777881044 2450300000
4 246.38742009496696 2447000000
5 245.85557518094379 2444400000
6 241.02558360648709 2444400000
7 239.3213472284815 2444300000
8 241.99643383989223 2446900000
9 241.33489341174754 2444400000
10 239.55507907227593 2444300000
1 246.54044214040516 2450300000
2 245.60644782648058 2450200000
3 244.05709961641855 2450300000
4 240.32765378948852 2446900000
5 238.76011088499922 2444300000
6 239.02883252576862 2444300000
7 237.96980800398615 2444300000
8 240.63136522723289 2446900000
9 242.29408037949452 2444300000
10 239.890048061985 2444400000
1 242.99714203338306 2450100000
2 246.75737888165145 2450500000
3 243.71466613668781 2450500000
4 241.37361545373369 2447200000
5 240.0990537825835 2444300000
6 238.3183063818592 2444300000
7 238.64394475904191 2444300000
8 238.95745333993955 2446800000
9 236.75123000799613 2444300000
10 236.62433367763288 2444300000
1 248.52692954734403 2450300000
2 246.42147683068947 2450300000
3 242.07294486261139 2450200000
4 238.3579614850978 2447000000
5 237.59051860477484 2444300000
6 238.28238352363087 2444300000
7 237.70435207760102 2444300000
8 240.35191338205777 2446900000
9 239.58353744048236 2444300000
10 238.55670353191681 2444400000
1 257.30516981248275 2450400000
2 244.30902615463992 2450400000
3 248.46441444341485 2450500000
4 241.15807830436643 2447200000
5 241.42026851636729 2444300000
6 244.43265677061913 2444600000
7 239.75895295598457 2444400000
8 245.52060619123449 2446900000
9 241.59335137873813 2444300000
10 248.25587525344281 2444400000
Raw
CscBenchRawData.csv
Iteration Group Time Instructions
0 1 1955.964641 6797200000
1 1 531.821121 2398200000
2 1 539.2244955 2380200000
3 1 534.4201631 2353200000
4 1 529.1273731 2357800000
5 1 533.6219292 2362600000
6 1 533.4264528 2356200000
7 1 527.2416563 2360500000
8 1 535.5482341 2357700000
9 1 527.4473963 2356600000
10 1 533.7777504 2362000000
0 2 1595.550138 5205100000
1 2 521.356839 1353400000
2 2 502.2967303 1384000000
3 2 526.6836857 1342900000
4 2 498.3372848 1381600000
5 2 500.6676053 1388600000
6 2 524.1779497 1338100000
7 2 518.8571679 1386900000
8 2 504.8621822 1381800000
9 2 496.9787477 1376300000
10 2 512.9163669 1372100000
0 1 1716.200556 6788100000
1 1 531.2444891 2398200000
2 1 538.8036848 2380200000
3 1 530.9281813 2353400000
4 1 544.8331267 2357600000
5 1 531.9834736 2362600000
6 1 524.4891256 2356300000
7 1 551.7788346 2360300000
8 1 531.9064961 2357000000
9 1 527.48052 2356400000
10 1 524.1298971 2362100000
0 2 1519.162279 5194900000
1 2 526.6402984 1359400000
2 2 506.4712463 1367000000
3 2 507.8031913 1377400000
4 2 514.5954106 1372200000
5 2 508.2011419 1370000000
6 2 512.2333661 1375400000
7 2 501.7648854 1340900000
8 2 482.6791174 1369900000
9 2 497.4527428 1388900000
10 2 507.4402304 1356800000
0 1 1704.283498 6787100000
1 1 520.6248525 2398300000
2 1 526.7443347 2380400000
3 1 521.0937157 2353900000
4 1 522.8907917 2357300000
5 1 518.2086903 2362200000
6 1 522.591279 2356700000
7 1 524.2316007 2360000000
8 1 522.7340374 2357600000
9 1 517.3400103 2356600000
10 1 520.4102484 2361700000
0 2 1525.006042 5191700000
1 2 495.0141872 1390000000
2 2 495.7485064 1380700000
3 2 504.0158956 1386900000
4 2 492.1352267 1372100000
5 2 509.2671644 1378400000
6 2 524.4597342 1346800000
7 2 513.4687392 1375800000
8 2 513.0129388 1357600000
9 2 518.104654 1352500000
10 2 529.978325 1431300000
0 1 1713.550662 6789400000
1 1 527.2206625 2398200000
2 1 524.1443595 2380200000
3 1 523.191704 2353400000
4 1 550.6983497 2357400000
5 1 524.616955 2362400000
6 1 534.1836321 2356400000
7 1 525.0951489 2360300000
8 1 523.1828399 2357000000
9 1 523.8747048 2356400000
10 1 526.1984939 2361900000
0 2 1512.322007 5188400000
1 2 527.0816363 1353300000
2 2 515.514476 1352600000
3 2 501.0077062 1385500000
4 2 507.5960517 1344700000
5 2 517.8797863 1364600000
6 2 494.9446741 1378500000
7 2 499.2152955 1386800000
8 2 524.7863056 1339800000
9 2 504.7007626 1389100000
10 2 536.8125321 1388900000
0 1 1706.691729 6793400000
1 1 521.8994141 2398400000
2 1 524.4588011 2380200000
3 1 521.2075492 2353400000
4 1 524.4009513 2357200000
5 1 524.783973 2362100000
6 1 526.5497914 2356800000
7 1 524.8842771 2359900000
8 1 518.5534565 2356900000
9 1 519.4277349 2356400000
10 1 522.567486 2361600000
0 2 1522.962171 5167300000
1 2 515.9940695 1399800000
2 2 523.199635 1356100000
3 2 524.2143391 1360500000
4 2 501.4499772 1384700000
5 2 505.3063193 1395000000
6 2 525.4492457 1372200000
7 2 521.6110982 1357300000
8 2 520.2922161 1339300000
9 2 509.9590293 1395000000
10 2 519.7281806 1405300000
0 1 1726.681166 6792500000
1 1 544.1645883 2398700000
2 1 529.6601511 2380400000
3 1 522.4555186 2353700000
4 1 522.3720097 2357000000
5 1 523.0736717 2362400000
6 1 524.7069954 2356100000
7 1 522.6365325 2359900000
8 1 519.8485455 2357200000
9 1 526.1075204 2356200000
10 1 526.8567686 2361700000
0 2 1544.897974 5149000000
1 2 505.9254055 1380600000
2 2 512.9634865 1365700000
3 2 488.7449486 1374200000
4 2 514.067298 1373200000
5 2 502.7539303 1363000000
6 2 506.2477781 1364600000
7 2 500.3107094 1375900000
8 2 499.7303453 1386800000
9 2 514.5249645 1371600000
10 2 524.2614587 1338200000
0 1 1702.304008 6791700000
1 1 525.2486375 2398200000
2 1 519.6623998 2380100000
3 1 522.2180545 2353600000
4 1 525.9605632 2357300000
5 1 518.8030504 2362100000
6 1 521.6241611 2356500000
7 1 521.1249733 2360000000
8 1 518.6313671 2357300000
9 1 523.2369574 2356600000
10 1 525.7842147 2361700000
0 2 1535.571561 5191600000
1 2 501.6571168 1397700000
2 2 505.1537638 1387200000
3 2 520.9710182 1344800000
4 2 504.973683 1384000000
5 2 535.8160227 1368800000
6 2 496.640513 1364400000
7 2 496.1245301 1379200000
8 2 528.3016139 1342300000
9 2 528.4583682 1340600000
10 2 527.9353874 1337900000
0 1 1736.691981 6793400000
1 1 524.2446636 2398200000
2 1 523.2490872 2380100000
3 1 524.2199375 2353500000
4 1 523.3797158 2357300000
5 1 522.7876884 2362300000
6 1 522.0165133 2356500000
7 1 523.4912166 2360400000
8 1 522.6752546 2357700000
9 1 526.31466 2356800000
10 1 522.1070203 2362100000
0 2 1530.073031 5181200000
1 2 506.954572 1402300000
2 2 526.124782 1349300000
3 2 507.9688096 1381700000
4 2 526.5138686 1352100000
5 2 507.3436586 1371100000
6 2 504.0280254 1383400000
7 2 505.47567 1362300000
8 2 511.935253 1377900000
9 2 534.0040178 1358100000
10 2 527.52624 1401300000
0 1 1714.776238 6795400000
1 1 520.6295178 2398300000
2 1 521.2458047 2380100000
3 1 531.7287479 2353500000
4 1 521.0074076 2357300000
5 1 534.9128194 2362500000
6 1 517.2858928 2356200000
7 1 526.0804616 2360000000
8 1 518.2768038 2357100000
9 1 518.1755667 2356300000
10 1 517.343276 2361700000
0 2 1521.070856 5190500000
1 2 512.2496947 1368100000
2 2 502.7338695 1386900000
3 2 518.0505365 1350400000
4 2 515.9586131 1373700000
5 2 521.9633288 1372700000
6 2 519.2751794 1366400000
7 2 501.538618 1377900000
8 2 502.6928148 1387400000
9 2 501.3529388 1370100000
10 2 504.1861793 1360000000
0 1 1697.168906 6795200000
1 1 520.5152178 2398000000
2 1 520.0332916 2379900000
3 1 519.9521153 2353400000
4 1 522.688784 2357600000
5 1 525.584073 2362500000
6 1 522.4615835 2356400000
7 1 519.1034961 2359800000
8 1 525.7767502 2357400000
9 1 518.89589 2356100000
10 1 531.349925 2361800000
0 2 1519.849945 5176700000
1 2 517.1361364 1368700000
2 2 527.0984314 1340000000
3 2 516.2184707 1378100000
4 2 528.6795037 1335700000
5 2 526.6594261 1347800000
6 2 522.5315631 1351300000
7 2 508.7843052 1369600000
8 2 515.7538062 1373600000
9 2 528.2321009 1342200000
10 2 535.752108 1407200000
0 1 1695.843492 6785600000
1 1 526.3995686 2398500000
2 1 523.9596134 2380400000
3 1 524.1149681 2353400000
4 1 526.4989396 2357500000
5 1 526.3389196 2362800000
6 1 521.166028 2356500000
7 1 529.4273523 2360100000
8 1 527.5276396 2357000000
9 1 527.1315551 2356600000
10 1 528.1672531 2361800000
0 2 1541.706905 5147000000
1 2 508.5249141 1381100000
2 2 510.1815644 1365100000
3 2 491.8478438 1378700000
4 2 509.0833513 1379300000
5 2 512.1279302 1381700000
6 2 524.2721889 1337100000
7 2 514.6192037 1372700000
8 2 517.9567638 1346900000
9 2 498.644262 1382900000
10 2 516.2343327 1378400000
Raw
perftest.bat
@setlocal
@echo off
:SETUP
echo *** Setting up sandbox
rd /s /q sandbox
mkdir sandbox
pushd sandbox
set ITER=11
@rem stage stuff we need
xcopy /s C:\Repos\coreclr\packages\Microsoft.DotNet.xunit.performance.runner.Windows\1.0.0-alpha-build0029\tools\* . > NUL: 2>&1
xcopy /sy C:\Repos\coreclr\packages\Microsoft.DotNet.xunit.performance.analysis\1.0.0-alpha-build0029\tools\* . > NUL: 2>&1
xcopy /sy C:\Repos\coreclr\packages\xunit.console.netcore\1.0.2-prerelease-00129\runtimes\any\native\* . > NUL: 2>&1
xcopy /sy c:\repos\coreclr\bin\tests\Windows_NT.x64.Release\Tests\Core_Root\* . > NUL: 2>&1
@rem stage the test
xcopy /s c:\repos\coreclr\bin\tests\Windows_NT.x64.Release\jit\Performance\CodeQuality\Roslyn\CscBench\CscBench.exe > NUL: 2>&1
echo *** Doing %ITER% runs
for /L %%I in (1, 1, %ITER%) do (
call :DOIT %%I
)
popd
echo *** Done
goto :EOF
:DOIT
set CORE_ROOT=%CD%
xunit.performance.run.exe CscBench.exe -runner xunit.console.netcore.exe -runnerhost corerun.exe -verbose -runid perf-%1 > %1.log 2>&1
xunit.performance.analysis.exe perf-%1.xml -xml perf-summary-%1.xml > NUL: 2>&1
type perf-summary-%1.xml | findstr InstRetired >> IR.TXT
type perf-%1.xml | findstr InstRetired >> IRR.TXT
@AndyAyersMS
Copy link
Author

Sample output:

<?xml version="1.0" encoding="utf-8"?>
<results>
  <run id="perf">
    <test name="AddArray.Test">
      <summary>
        <Duration min="902" mean="941" max="1.01E+03" marginOfError="0.0253" stddev="35.4" />
        <InstRetired min="7.22E+09" mean="7.23E+09" max="7.24E+09" marginOfError="0.000734" stddev="7.9E+06" />
      </summary>
    </test>
  </run>
</results>

@AndyAyersMS
Copy link
Author

Note the above presumes your test case is from the CoreCLR repo, and you've both cleanly built (from the root of the repo, build x64 release) and run the tests once (tests\runtest.cmd x64 release). The latter is required to populate the right binaries into the Core_Root referenced by the last copy command.

@AndyAyersMS
Copy link
Author

Updated to handle some changes in the coreclr repo: packages moved, xunit perf version bumped

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment