Skip to content

Instantly share code, notes, and snippets.

@mpaperno
Last active September 3, 2018 22:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save mpaperno/6a981df2ec0c825c2b6fd2e80c169de0 to your computer and use it in GitHub Desktop.
Save mpaperno/6a981df2ec0c825c2b6fd2e80c169de0 to your computer and use it in GitHub Desktop.

All tests on STM32F4 @168MHz, with Kingston class 6 8GB SD. All times in micro seconds [us]. The "memory" statistic (where present) is delta of samples taken with luaGetMemUsed() before and after iteration/test (with no GC done in between).

The Lua environment was reloaded between each iteration (except for the tLdScr.lua test which is all one run).

NOTE: The times listed here are really only relevant in comparison to other tests run on the same system. They will vary greatly based on system load, SD card speed/integrity, and other factors. The percentage of change is the most relevant statistic here. The benchmarks were run under "ideal" conditions, using a high-priority thread, to ensure consistent results.

===========================================

!! tLdScr.lua test results

NOTE: "nb" or "nt" after the mode means the relevant file was removed before the test. So with "nb" the 
binary does not exist, and with "nt" the text/source version does not exist. This is to ensure proper 
behaviour in those situations.

++++ Loading with mode:         b
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, b): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, b): time = 7812 [us]; mem = 607 [B]
---- Loaded with mode:  b

++++ Loading with mode:         b-nb
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, b): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, b): time = 7570 [us]; mem = 155 [B]
cannot open /SCRIPTS/TELEMETRY/tIncFn.luac:
---- Loaded with mode:  b-nb

++++ Loading with mode:         t
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 1): res=4; ts=0:0
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, t): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, t): time = 91096 [us]; mem = 1186 [B]
---- Loaded with mode:  t

++++ Loading with mode:         t-nt
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=4; ts=0:0
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 1): res=0; ts=19739:43228
-E- luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, t): Error loading script: file not found.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, t): time = 14087 [us]; mem = 216 [B]
loadScript("/SCRIPTS/TELEMETRY/tIncFn.lua", "t") error: File not found
---- Loaded with mode:  t-nt

++++ Loading with mode:         T
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, T): loading /SCRIPTS/TELEMETRY/tIncFn.lua
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, T): time = 21420 [us]; mem = 1039 [B]
---- Loaded with mode:  T

++++ Loading with mode:         T-nt
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=4; ts=0:0
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, T): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, T): time = 21596 [us]; mem = 378 [B]
---- Loaded with mode:  T-nt

++++ Loading with mode:         bt
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, bt): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, bt): time = 20525 [us]; mem = 378 [B]
---- Loaded with mode:  bt

++++ Loading with mode:         bt-nb
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 1): res=4; ts=0:0
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, bt): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, bt): time = 95140 [us]; mem = 742 [B]
---- Loaded with mode:  bt-nb

++++ Loading with mode:         btx
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 0): res=0; ts=0:0
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 0): res=0; ts=0:0
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, btx): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, btx): time = 20560 [us]; mem = 491 [B]
---- Loaded with mode:  btx

++++ Loading with mode:         btc
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, btc): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, btc): time = 84300 [us]; mem = 1039 [B]
---- Loaded with mode:  btc

++++ Loading with mode:         bcd
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, bcd): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, bcd): time = 88912 [us]; mem = 1039 [B]
---- Loaded with mode:  bcd

++++ Loading with mode:         bx
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, bx): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, bx): time = 7721 [us]; mem = 378 [B]
---- Loaded with mode:  bx

++++ Loading with mode:         tx
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, tx): loading /SCRIPTS/TELEMETRY/tIncFn.lua
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, tx): time = 8562 [us]; mem = 1039 [B]
---- Loaded with mode:  tx

++++ Loading with mode:         Tx
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 0): res=0; ts=0:0
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, Tx): loading /SCRIPTS/TELEMETRY/tIncFn.lua
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, Tx): time = 22030 [us]; mem = 1039 [B]
---- Loaded with mode:  Tx

++++ Loading with mode:         Tx-nt
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 0): res=4; ts=0:0
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.luac, 0): res=0; ts=0:0
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, Tx): loading /SCRIPTS/TELEMETRY/tIncFn.luac
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, Tx): time = 21604 [us]; mem = 378 [B]
---- Loaded with mode:  Tx-nt

++++ Loading with mode:         tc
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, tc): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, tc): time = 85366 [us]; mem = 1039 [B]
---- Loaded with mode:  tc

++++ Loading with mode:         Tc
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, Tc): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, Tc): time = 88394 [us]; mem = 1039 [B]
---- Loaded with mode:  Tc

++++ Loading with mode:         tcd
-D- f_stat(/SCRIPTS/TELEMETRY/tIncFn.lua, 1): res=0; ts=19739:43228
luaLoadScriptFileToState(/SCRIPTS/TELEMETRY/tIncFn.lua, tcd): loading /SCRIPTS/TELEMETRY/tIncFn.lua
luaDumpState(/SCRIPTS/TELEMETRY/tIncFn.luac): Saved bytecode to file, ts=19739:43228.
           luaLoadScript(/SCRIPTS/TELEMETRY/tIncFn.lua, tcd): time = 86662 [us]; mem = 1039 [B]
---- Loaded with mode:  tcd

++ [benchmark] iter: 1; elapsed: 1107727;

========================================

loadScript() vs loadfile(), SMALL file, (b)in and (t)ext versions, 20 loops each  [10 iterations]
== [lS-b]: iterations: 10;  avg: 187292;  min: 186786;  max: 187371;  ttl: 1872920;
== [lf-b]: iterations: 10;  avg: 189974;  min: 189960;  max: 189990;  ttl: 1899745;
== [lS-t]: iterations: 10;  avg: 206405;  min: 206392;  max: 206422;  ttl: 2064057;
== [lf-t]: iterations: 10;  avg: 207218;  min: 207202;  max: 207241;  ttl: 2072184;
       test deltas :      avg (percent  chng);       total (percent  chng);
  [lf-b] vs [lS-b] :     2682 (  1.41% worse);       26825 (  1.41% worse);
  [lf-t] vs [lS-t] :      812 (  0.39% worse);        8127 (  0.39% worse);
  [lf-t] vs [lf-b] :    17243 (  8.32% worse);      172439 (  8.32% worse);
  [lS-t] vs [lS-b] :    19113 (  9.26% worse);      191137 (  9.26% worse);

loadScript() vs loadfile(), LARGE file, (b)in and (t)ext versions, 20 loops each  [10 iterations]
== [lS-b]: iterations: 10;  avg: 628826;  min: 628785;  max: 628862;  ttl: 6288261;
== [lf-b]: iterations: 10;  avg: 629271;  min: 629222;  max: 629329;  ttl: 6292711;
== [lS-t]: iterations: 10;  avg: 1811346;  min: 1810992;  max: 1814299;  ttl: 18113460;
== [lf-t]: iterations: 10;  avg: 1811274;  min: 1811203;  max: 1811343;  ttl: 18112745;
       test deltas :      avg (percent  chng);       total (percent  chng);
  [lf-b] vs [lS-b] :      445 (  0.07% worse);        4450 (  0.07% worse);
  [lf-t] vs [lS-t] :      -71 (  0.00% bettr);        -715 (  0.00% bettr);
  [lf-t] vs [lf-b] :  1182003 ( 65.26% worse);    11820034 ( 65.26% worse);
  [lS-t] vs [lS-b] :  1182519 ( 65.27% worse);    11825199 ( 65.27% worse);

No-luac version of loadScript() vs loadfile(), SMALL file, (b)in and (t)ext versions, 20 loops each  [10 iterations]
== [lS-b]: iterations: 10;  avg: 203892;  min: 203367;  max: 203966;  ttl: 2038927;
== [lf-b]: iterations: 10;  avg: 189935;  min: 189926;  max: 189944;  ttl: 1899354;
== [lS-t]: iterations: 10;  avg: 206097;  min: 206083;  max: 206107;  ttl: 2060973;
== [lf-t]: iterations: 10;  avg: 207223;  min: 207206;  max: 207243;  ttl: 2072234;
       test deltas :      avg (percent  chng);       total (percent  chng);
  [lf-b] vs [lS-b] :   -13957 (  7.34% bettr);     -139573 (  7.34% bettr);
  [lf-t] vs [lS-t] :     1126 (  0.54% worse);       11261 (  0.54% worse);
  [lf-t] vs [lf-b] :    17288 (  8.34% worse);      172880 (  8.34% worse);
  [lS-t] vs [lS-b] :     2204 (  1.07% worse);       22046 (  1.07% worse);

No-luac version of loadScript() vs loadfile(), LARGE file, (b)in and (t)ext versions, 20 loops each  [10 iterations]
== [lS-b]: iterations: 10;  avg: 628572;  min: 628528;  max: 628616;  ttl: 6285721;
== [lf-b]: iterations: 10;  avg: 629280;  min: 629252;  max: 629326;  ttl: 6292802;
== [lS-t]: iterations: 10;  avg: 1810864;  min: 1809453;  max: 1812676;  ttl: 18108648;
== [lf-t]: iterations: 10;  avg: 1812030;  min: 1811950;  max: 1812094;  ttl: 18120303;
       test deltas :      avg (percent  chng);       total (percent  chng);
  [lf-b] vs [lS-b] :      708 (  0.11% worse);        7081 (  0.11% worse);
  [lf-t] vs [lS-t] :     1165 (  0.06% worse);       11655 (  0.06% worse);
  [lf-t] vs [lf-b] :  1182750 ( 65.27% worse);    11827501 ( 65.27% worse);
  [lS-t] vs [lS-b] :  1182292 ( 65.29% worse);    11822927 ( 65.29% worse);

Previous version of loadScript() vs loadfile(), SMALL file, (b)in and (t)ext versions, 20 loops each  [10 iterations]
== [lS-b]: iterations: 10;  avg: 439395;  min: 439378;  max: 439439;  ttl: 4393951;
== [lf-b]: iterations: 10;  avg: 189944;  min: 189935;  max: 189957;  ttl: 1899443;
== [lS-t]: iterations: 10;  avg: 459414;  min: 459405;  max: 459432;  ttl: 4594148;
== [lf-t]: iterations: 10;  avg: 207239;  min: 207225;  max: 207265;  ttl: 2072399;
       test deltas :      avg (percent  chng);       total (percent  chng);
  [lf-b] vs [lS-b] :  -249450 (131.33% bettr);    -2494508 (131.33% bettr);
  [lf-t] vs [lS-t] :  -252174 (121.68% bettr);    -2521749 (121.68% bettr);
  [lf-t] vs [lf-b] :    17295 (  8.35% worse);      172956 (  8.35% worse);
  [lS-t] vs [lS-b] :    20019 (  4.36% worse);      200197 (  4.36% worse);

Previous version of loadScript() vs loadfile(), LARGE file, (b)in and (t)ext versions, 20 loops each  [10 iterations]
== [lS-b]: iterations: 10;  avg: 755166;  min: 754457;  max: 755277;  ttl: 7551667;
== [lf-b]: iterations: 10;  avg: 629258;  min: 629219;  max: 629306;  ttl: 6292583;
== [lS-t]: iterations: 10;  avg: 1934185;  min: 1934111;  max: 1934522;  ttl: 19341855;
== [lf-t]: iterations: 10;  avg: 1810676;  min: 1810404;  max: 1810743;  ttl: 18106762;
       test deltas :      avg (percent  chng);       total (percent  chng);
  [lf-b] vs [lS-b] :  -125908 ( 20.01% bettr);    -1259084 ( 20.01% bettr);
  [lf-t] vs [lS-t] :  -123509 (  6.82% bettr);    -1235093 (  6.82% bettr);
  [lf-t] vs [lf-b] :  1181417 ( 65.25% worse);    11814179 ( 65.25% worse);
  [lS-t] vs [lS-b] :  1179018 ( 60.95% worse);    11790188 ( 60.95% worse);

========================================

luaLoadScriptToFile() benchmarks

luaLoadScriptToFile various modes, SMALL file (NC=no luac, PV=prev. version)  [10 iterations]

== [   t ]: iterations: 10;  avg: 13779;  min: 13777;  max: 13786;  ttl: 137792;  avg memory: 1334
== [   T ]: iterations: 10;  avg: 13776;  min: 13774;  max: 13782;  ttl: 137760;  avg memory: 1334
== [   bt]: iterations: 10;  avg: 15015;  min: 15015;  max: 15022;  ttl: 150159;  avg memory: 656
== [   b ]: iterations: 10;  avg: 6607;  min: 6607;  max: 6614;  ttl: 66079;  avg memory: 656
== [   tx]: iterations: 10;  avg: 5369;  min: 5367;  max: 5388;  ttl: 53692;  avg memory: 1334
== [   tc]: iterations: 10;  avg: 78048;  min: 76545;  max: 79554;  ttl: 780486;  avg memory: 1334
== [NC-t ]: iterations: 10;  avg: 5356;  min: 5354;  max: 5375;  ttl: 53565;  avg memory: 1334
== [NC-b ]: iterations: 10;  avg: 6594;  min: 6593;  max: 6601;  ttl: 65948;  avg memory: 656
== [PV-t ]: iterations: 10;  avg: 13796;  min: 13794;  max: 13802;  ttl: 137963;  avg memory: 1334
== [PV-bt]: iterations: 10;  avg: 15033;  min: 15033;  max: 15039;  ttl: 150338;  avg memory: 656
== [PV-b ]: iterations: 10;  avg: 15033;  min: 15033;  max: 15037;  ttl: 150336;  avg memory: 656
== [PV-tx]: iterations: 10;  avg: 13794;  min: 13792;  max: 13801;  ttl: 137948;  avg memory: 1334

Comparison report:
         test deltas :      avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [   T ] vs [   t ] :       -3 (  0.02% bettr);        -32 (  0.02% bettr);          0 (  0.00%  n/c );
  [   bt] vs [   t ] :     1236 (  8.23% worse);      12367 (  8.24% worse);       -678 (103.34% bettr);
  [   b ] vs [   t ] :    -7171 (108.54% bettr);     -71713 (108.52% bettr);       -678 (103.34% bettr);
  [   tx] vs [   t ] :    -8410 (156.63% bettr);     -84100 (156.63% bettr);          0 (  0.00%  n/c );
  [   tc] vs [   t ] :    64269 ( 82.34% worse);     642694 ( 82.34% worse);          0 (  0.00%  n/c );
  [   bt] vs [   T ] :     1239 (  8.25% worse);      12399 (  8.26% worse);       -678 (103.34% bettr);
  [   b ] vs [   T ] :    -7168 (108.48% bettr);     -71681 (108.48% bettr);       -678 (103.34% bettr);
  [   tx] vs [   T ] :    -8406 (156.57% bettr);     -84068 (156.57% bettr);          0 (  0.00%  n/c );
  [   tc] vs [   T ] :    64272 ( 82.34% worse);     642726 ( 82.34% worse);          0 (  0.00%  n/c );
  [   b ] vs [   bt] :    -8408 (127.26% bettr);     -84080 (127.23% bettr);          0 (  0.00%  n/c );
  [   tx] vs [   bt] :    -9646 (179.66% bettr);     -96467 (179.66% bettr);        678 ( 50.81% worse);
  [   tc] vs [   bt] :    63032 ( 80.76% worse);     630327 ( 80.76% worse);        678 ( 50.81% worse);
  [   tx] vs [   b ] :    -1238 ( 23.06% bettr);     -12387 ( 23.06% bettr);        678 ( 50.81% worse);
  [   tc] vs [   b ] :    71440 ( 91.52% worse);     714407 ( 91.52% worse);        678 ( 50.81% worse);
  [   tc] vs [   tx] :    72679 ( 93.12% worse);     726794 ( 93.12% worse);          0 (  0.00%  n/c );

     No-luac vs.            avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [NC-t ] vs [   t ] :    -8422 (157.24% bettr);     -84227 (157.24% bettr);          0 (  0.00%  n/c );
  [NC-t ] vs [   bt] :    -9659 (180.33% bettr);     -96594 (180.33% bettr);        678 ( 50.81% worse);
  [NC-b ] vs [   bt] :    -8421 (127.70% bettr);     -84211 (127.69% bettr);          0 (  0.00%  n/c );
  [NC-b ] vs [   b ] :      -13 (  0.20% bettr);       -131 (  0.20% bettr);          0 (  0.00%  n/c );
  [NC-t ] vs [   tx] :      -12 (  0.22% bettr);       -127 (  0.24% bettr);          0 (  0.00%  n/c );
  [NC-b ] vs [   tx] :     1225 ( 18.57% worse);      12256 ( 18.57% worse);       -678 (103.34% bettr);
  [NC-b ] vs [NC-t ] :     1238 ( 18.77% worse);      12383 ( 18.78% worse);       -678 (103.34% bettr);

     Previous vs.           avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [PV-t ] vs [   t ] :       17 (  0.12% worse);        171 (  0.12% worse);          0 (  0.00%  n/c );
  [PV-bt] vs [   bt] :       17 (  0.11% worse);        179 (  0.12% worse);          0 (  0.00%  n/c );
  [PV-b ] vs [   b ] :     8425 ( 56.04% worse);      84257 ( 56.04% worse);          0 (  0.00%  n/c );
  [PV-tx] vs [   tx] :     8425 ( 61.08% worse);      84256 ( 61.08% worse);          0 (  0.00%  n/c );
  [PV-t ] vs [NC-t ] :     8439 ( 61.17% worse);      84398 ( 61.17% worse);          0 (  0.00%  n/c );
  [PV-tx] vs [NC-t ] :     8438 ( 61.17% worse);      84383 ( 61.17% worse);          0 (  0.00%  n/c );
  [PV-t ] vs [NC-b ] :     7201 ( 52.20% worse);      72015 ( 52.20% worse);        678 ( 50.81% worse);
  [PV-bt] vs [NC-b ] :     8439 ( 56.13% worse);      84390 ( 56.13% worse);          0 (  0.00%  n/c );
  [PV-b ] vs [NC-b ] :     8438 ( 56.13% worse);      84388 ( 56.13% worse);          0 (  0.00%  n/c );
  [PV-bt] vs [PV-t ] :     1237 (  8.23% worse);      12375 (  8.23% worse);       -678 (103.34% bettr);
  [PV-b ] vs [PV-t ] :     1237 (  8.23% worse);      12373 (  8.23% worse);       -678 (103.34% bettr);
  [PV-tx] vs [PV-t ] :       -1 (  0.01% bettr);        -15 (  0.01% bettr);          0 (  0.00%  n/c );
  [PV-b ] vs [PV-bt] :        0 (  0.00% worse);         -2 (  0.00% bettr);          0 (  0.00%  n/c );
  [PV-tx] vs [PV-bt] :    -1239 (  8.98% bettr);     -12390 (  8.98% bettr);        678 ( 50.81% worse);
  [PV-tx] vs [PV-b ] :    -1238 (  8.97% bettr);     -12388 (  8.98% bettr);        678 ( 50.81% worse);

luaLoadScriptToFile various modes, LARGE file (NC=no luac, PV=prev. version)  [10 iterations]

== [   t ]: iterations: 10;  avg: 95201;  min: 95185;  max: 95235;  ttl: 952010;  avg memory: 40135
== [   T ]: iterations: 10;  avg: 95187;  min: 95187;  max: 95189;  ttl: 951878;  avg memory: 40135
== [   bt]: iterations: 10;  avg: 36693;  min: 36693;  max: 36694;  ttl: 366932;  avg memory: 15245
== [   b ]: iterations: 10;  avg: 30401;  min: 30395;  max: 30413;  ttl: 304019;  avg memory: 15245
== [   tx]: iterations: 10;  avg: 88830;  min: 88813;  max: 88843;  ttl: 888309;  avg memory: 40135
== [   tc]: iterations: 10;  avg: 349205;  min: 312830;  max: 481315;  ttl: 3492059;  avg memory: 40135
== [NC-t ]: iterations: 10;  avg: 88848;  min: 88833;  max: 88862;  ttl: 888482;  avg memory: 40135
== [NC-b ]: iterations: 10;  avg: 30385;  min: 30379;  max: 30401;  ttl: 303854;  avg memory: 15245
== [PV-t ]: iterations: 10;  avg: 95135;  min: 95133;  max: 95137;  ttl: 951351;  avg memory: 40135
== [PV-bt]: iterations: 10;  avg: 36718;  min: 36718;  max: 36719;  ttl: 367184;  avg memory: 15245
== [PV-b ]: iterations: 10;  avg: 36716;  min: 36716;  max: 36717;  ttl: 367164;  avg memory: 15245
== [PV-tx]: iterations: 10;  avg: 95118;  min: 95118;  max: 95120;  ttl: 951189;  avg memory: 40135

Comparison report: 
         test deltas :      avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [   T ] vs [   t ] :      -13 (  0.01% bettr);       -132 (  0.01% bettr);          0 (  0.00%  n/c );
  [   bt] vs [   t ] :   -58507 (159.44% bettr);    -585078 (159.44% bettr);     -24890 (163.27% bettr);
  [   b ] vs [   t ] :   -64799 (213.15% bettr);    -647991 (213.13% bettr);     -24890 (163.27% bettr);
  [   tx] vs [   t ] :    -6370 (  7.17% bettr);     -63701 (  7.17% bettr);          0 (  0.00%  n/c );
  [   tc] vs [   t ] :   254004 ( 72.74% worse);    2540049 ( 72.74% worse);          0 (  0.00%  n/c );
  [   bt] vs [   T ] :   -58494 (159.41% bettr);    -584946 (159.41% bettr);     -24890 (163.27% bettr);
  [   b ] vs [   T ] :   -64785 (213.10% bettr);    -647859 (213.10% bettr);     -24890 (163.27% bettr);
  [   tx] vs [   T ] :    -6356 (  7.15% bettr);     -63569 (  7.15% bettr);          0 (  0.00%  n/c );
  [   tc] vs [   T ] :   254018 ( 72.74% worse);    2540181 ( 72.74% worse);          0 (  0.00%  n/c );
  [   b ] vs [   bt] :    -6291 ( 20.69% bettr);     -62913 ( 20.69% bettr);          0 (  0.00%  n/c );
  [   tx] vs [   bt] :    52137 ( 58.69% worse);     521377 ( 58.69% worse);      24890 ( 62.02% worse);
  [   tc] vs [   bt] :   312512 ( 89.49% worse);    3125127 ( 89.49% worse);      24890 ( 62.02% worse);
  [   tx] vs [   b ] :    58429 ( 65.77% worse);     584290 ( 65.77% worse);      24890 ( 62.02% worse);
  [   tc] vs [   b ] :   318804 ( 91.29% worse);    3188040 ( 91.29% worse);      24890 ( 62.02% worse);
  [   tc] vs [   tx] :   260375 ( 74.56% worse);    2603750 ( 74.56% worse);          0 (  0.00%  n/c );

     No-luac vs.            avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [NC-t ] vs [   t ] :    -6352 (  7.15% bettr);     -63528 (  7.15% bettr);          0 (  0.00%  n/c );
  [NC-t ] vs [   bt] :    52155 ( 58.70% worse);     521550 ( 58.70% worse);      24890 ( 62.02% worse);
  [NC-b ] vs [   bt] :    -6307 ( 20.76% bettr);     -63078 ( 20.76% bettr);          0 (  0.00%  n/c );
  [NC-b ] vs [   b ] :      -16 (  0.05% bettr);       -165 (  0.05% bettr);          0 (  0.00%  n/c );
  [NC-t ] vs [   tx] :       17 (  0.02% worse);        173 (  0.02% worse);          0 (  0.00%  n/c );
  [NC-b ] vs [NC-t ] :   -58462 (192.40% bettr);    -584628 (192.40% bettr);     -24890 (163.27% bettr);

     Previous vs.           avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [PV-t ] vs [   t ] :      -65 (  0.07% bettr);       -659 (  0.07% bettr);          0 (  0.00%  n/c );
  [PV-bt] vs [   bt] :       25 (  0.07% worse);        252 (  0.07% worse);          0 (  0.00%  n/c );
  [PV-b ] vs [   b ] :     6314 ( 17.20% worse);      63145 ( 17.20% worse);          0 (  0.00%  n/c );
  [PV-tx] vs [   tx] :     6288 (  6.61% worse);      62880 (  6.61% worse);          0 (  0.00%  n/c );
  [PV-t ] vs [NC-t ] :     6286 (  6.61% worse);      62869 (  6.61% worse);          0 (  0.00%  n/c );
  [PV-tx] vs [NC-t ] :     6270 (  6.59% worse);      62707 (  6.59% worse);          0 (  0.00%  n/c );
  [PV-t ] vs [NC-b ] :    64749 ( 68.06% worse);     647497 ( 68.06% worse);      24890 ( 62.02% worse);
  [PV-bt] vs [NC-b ] :     6333 ( 17.25% worse);      63330 ( 17.25% worse);          0 (  0.00%  n/c );
  [PV-b ] vs [NC-b ] :     6331 ( 17.23% worse);      63310 ( 17.23% worse);          0 (  0.00%  n/c );
  [PV-bt] vs [PV-t ] :   -58416 (159.09% bettr);    -584167 (159.09% bettr);     -24890 (163.27% bettr);
  [PV-b ] vs [PV-t ] :   -58418 (159.11% bettr);    -584187 (159.11% bettr);     -24890 (163.27% bettr);
  [PV-tx] vs [PV-t ] :      -16 (  0.02% bettr);       -162 (  0.02% bettr);          0 (  0.00%  n/c );
  [PV-b ] vs [PV-bt] :       -2 (  0.01% bettr);        -20 (  0.01% bettr);          0 (  0.00%  n/c );
  [PV-tx] vs [PV-bt] :    58400 ( 61.40% worse);     584005 ( 61.40% worse);      24890 ( 62.02% worse);
  [PV-tx] vs [PV-b ] :    58402 ( 61.40% worse);     584025 ( 61.40% worse);      24890 ( 62.02% worse);

=====================================

No-luac luaLoadScriptToFile (lS) vs. luaL_loadfilex (lF), (t)ext and (b)inary  [10 iterations]
== [lS-t]: iterations: 10;  avg: 88735;  min: 88720;  max: 88746;  ttl: 887359;  avg memory: 40135
== [lS-b]: iterations: 10;  avg: 30311;  min: 30300;  max: 30351;  ttl: 303113;  avg memory: 15245
== [lF-t]: iterations: 10;  avg: 88697;  min: 88684;  max: 88730;  ttl: 886978;  avg memory: 40135
== [lF-b]: iterations: 10;  avg: 30304;  min: 30292;  max: 30351;  ttl: 303045;  avg memory: 15245
       test deltas :      avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [lF-t] vs [lS-t] :      -38 (  0.04% bettr);       -381 (  0.04% bettr);          0 (  0.00%  n/c );
  [lF-b] vs [lS-b] :       -6 (  0.02% bettr);        -68 (  0.02% bettr);          0 (  0.00%  n/c );
  [lS-b] vs [lS-t] :   -58424 (192.75% bettr);    -584246 (192.75% bettr);     -24890 (163.27% bettr);
  [lF-b] vs [lS-t] :   -58431 (192.82% bettr);    -584314 (192.80% bettr);     -24890 (163.27% bettr);
  [lF-t] vs [lS-b] :    58386 ( 65.83% worse);     583865 ( 65.83% worse);      24890 ( 62.02% worse);
  [lF-b] vs [lF-t] :   -58393 (192.69% bettr);    -583933 (192.69% bettr);     -24890 (163.27% bettr);

Comparison to using malloc() (BIN_ALLOCATOR disabled)

malloc() (ML) No-luac luaLoadScriptToFile (lS) vs. luaL_loadfilex (lF), (t)ext and (b)inary  [10 iterations]
== [lS-t]: iterations: 10;  avg: 83549;  min: 83331;  max: 84483;  ttl: 835490;  avg memory: 40135
== [lS-b]: iterations: 10;  avg: 28458;  min: 28451;  max: 28486;  ttl: 284586;  avg memory: 15245
== [lF-t]: iterations: 10;  avg: 83337;  min: 83321;  max: 83352;  ttl: 833373;  avg memory: 40135
== [lF-b]: iterations: 10;  avg: 28458;  min: 28451;  max: 28476;  ttl: 284580;  avg memory: 15245
       test deltas :      avg (percent  chng);      total (percent  chng);     memory (percent  chng);
  [lF-t] vs [lS-t] :     -211 (  0.25% bettr);      -2117 (  0.25% bettr);          0 (  0.00%  n/c );
  [lF-b] vs [lS-b] :        0 (  0.00%  n/c );         -6 (  0.00% bettr);          0 (  0.00%  n/c );

    BA vs. malloc deltas :      avg (percent  chng); 
  [BA-lS-t] vs [ML-lS-t] :     5186 (  6.21% worse);
  [BA-lS-b] vs [ML-lS-b] :    1,853 (  6.51% worse);
  [BA-lF-t] vs [ML-lF-t] :     5356 (  6.43% worse);
  [BA-lF-b] vs [ML-lF-b] :     1846 (  6.49% worse);

=========================================

f_stat with vs. w/out FILINFO  [10 iterations]
== [with  (1 file)]: iterations: 10;  avg: 6285;  min: 6285;  max: 6286;  ttl: 62854;
== [w/out (1 file)]: iterations: 10;  avg: 6283;  min: 6282;  max: 6287;  ttl: 62833;
== [with  (3 fils)]: iterations: 10;  avg: 13629;  min: 13628;  max: 13633;  ttl: 136291;
== [w/out (3 fils)]: iterations: 10;  avg: 13619;  min: 13618;  max: 13624;  ttl: 136194;
                           test deltas :      avg (percent  chng);           total (percent  chng);
  [w/out (1 file)] vs [with  (1 file)] :       -2 (  0.03% bettr);             -21 (  0.03% bettr);
  [w/out (3 fils)] vs [with  (3 fils)] :       -9 (  0.07% bettr);             -97 (  0.07% bettr);
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment