Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

@brettcannon

brettcannon/3.5-comparison.txt

Created September 13, 2016 18:53
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save brettcannon/b57a86a41b4eb0ea64c3b837690c9662 to your computer and use it in GitHub Desktop.
Save brettcannon/b57a86a41b4eb0ea64c3b837690c9662 to your computer and use it in GitHub Desktop.
Download ZIP
Performance comparison between Python 3.5.2+ & 3.6b1+
Raw
3.5-comparison.txt
+------------------------------------+---------------+---------------+--------------+-------------------------+
| Benchmark | 3.5-perf.json | 3.6-perf.json | Change | Significance |
+====================================+===============+===============+==============+=========================+
| 2to3 | 0.68 | 0.62 | 1.09x faster | Significant (t=65.03) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| call_method | 0.02 | 0.02 | 1.07x faster | Significant (t=46.01) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| call_method_slots | 0.02 | 0.02 | 1.06x faster | Significant (t=29.45) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| call_method_unknown | 0.02 | 0.02 | 1.07x faster | Significant (t=39.01) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| chaos | 0.0 | 0.0 | 1.11x faster | Significant (t=32.86) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| django_template | 0.7 | 0.67 | 1.05x faster | Significant (t=18.01) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| elementtree/generate | 0.24 | 0.22 | 1.12x faster | Significant (t=37.47) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| elementtree/iterparse | 0.41 | 0.2 | 2.03x faster | Significant (t=245.94) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| elementtree/parse | 0.27 | 0.24 | 1.16x faster | Significant (t=32.72) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| elementtree/process | 0.2 | 0.17 | 1.13x faster | Significant (t=29.71) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| fannkuch | 1.25 | 0.98 | 1.28x faster | Significant (t=84.36) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| fastpickle/pickle | 0.0 | 0.0 | 1.08x faster | Significant (t=11.73) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| fastpickle/pickle_list | 0.0 | 0.0 | 1.05x faster | Significant (t=13.73) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| fastpickle/unpickle | 0.0 | 0.0 | 1.38x faster | Significant (t=22.84) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| fastpickle/unpickle_list | 0.0 | 0.0 | 2.26x faster | Significant (t=129.58) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| float | 0.3 | 0.28 | 1.07x faster | Significant (t=23.95) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| go | 0.54 | 0.53 | 1.02x faster | Significant (t=8.43) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| hexiom2/level_25 | 0.02 | 0.02 | 1.03x faster | Significant (t=14.82) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| json_dump_v2 | 0.03 | 0.02 | 1.15x faster | Significant (t=39.51) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| logging/no_output | 0.0 | 0.0 | 1.04x faster | Significant (t=8.31) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| mako | 0.04 | 0.04 | 1.06x slower | Significant (t=-22.74) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| meteor_contest | 0.21 | 0.19 | 1.07x faster | Significant (t=16.66) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| nbody | 0.26 | 0.23 | 1.12x faster | Significant (t=45.25) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| nqueens | 0.24 | 0.2 | 1.16x faster | Significant (t=43.43) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pathlib | 0.05 | 0.05 | 1.02x slower | Significant (t=-5.18) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.BuiltinFunctionCalls | 0.0 | 0.0 | 1.08x slower | Significant (t=-32.64) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.BuiltinMethodLookup | 0.0 | 0.0 | 1.23x faster | Significant (t=54.90) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareFloats | 0.0 | 0.0 | 1.20x faster | Significant (t=50.87) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareFloatsIntegers | 0.0 | 0.0 | 1.58x faster | Significant (t=80.25) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareIntegers | 0.0 | 0.0 | 1.24x faster | Significant (t=67.17) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareInternedStrings | 0.0 | 0.0 | 1.25x faster | Significant (t=36.71) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareLongs | 0.0 | 0.0 | 1.23x faster | Significant (t=78.35) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareStrings | 0.0 | 0.0 | 1.13x faster | Significant (t=44.88) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CompareUnicode | 0.0 | 0.0 | 1.13x faster | Significant (t=43.82) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.ComplexPythonFunctionCalls | 0.0 | 0.0 | 1.25x slower | Significant (t=-62.75) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.ConcatStrings | 0.0 | 0.0 | 1.07x faster | Significant (t=16.46) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.ConcatUnicode | 0.0 | 0.0 | 1.08x faster | Significant (t=19.27) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CreateInstances | 0.0 | 0.0 | 1.14x faster | Significant (t=29.49) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CreateNewInstances | 0.0 | 0.0 | 1.13x faster | Significant (t=26.90) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CreateStringsWithConcat | 0.0 | 0.0 | 1.16x faster | Significant (t=47.73) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.CreateUnicodeWithConcat | 0.0 | 0.0 | 1.16x faster | Significant (t=57.42) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.DictCreation | 0.0 | 0.0 | 1.93x faster | Significant (t=123.84) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.DictWithFloatKeys | 0.0 | 0.0 | 1.13x faster | Significant (t=40.70) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.DictWithIntegerKeys | 0.0 | 0.0 | 1.16x faster | Significant (t=36.43) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.DictWithStringKeys | 0.0 | 0.0 | 1.17x faster | Significant (t=21.18) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.ForLoops | 0.0 | 0.0 | 1.50x faster | Significant (t=137.07) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.IfThenElse | 0.0 | 0.0 | 1.48x faster | Significant (t=45.83) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.NestedForLoops | 0.0 | 0.0 | 1.28x faster | Significant (t=82.12) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.NestedListComprehensions | 0.0 | 0.0 | 1.54x faster | Significant (t=82.01) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.NormalClassAttribute | 0.0 | 0.0 | 1.06x faster | Significant (t=7.15) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.NormalInstanceAttribute | 0.0 | 0.0 | 1.10x faster | Significant (t=6.00) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SecondImport | 0.0 | 0.0 | 3.02x faster | Significant (t=269.54) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SecondPackageImport | 0.0 | 0.0 | 2.80x faster | Significant (t=226.09) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SecondSubmoduleImport | 0.0 | 0.0 | 2.96x faster | Significant (t=204.35) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleComplexArithmetic | 0.0 | 0.0 | 1.20x faster | Significant (t=48.87) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleDictManipulation | 0.0 | 0.0 | 1.03x slower | Significant (t=-10.36) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleFloatArithmetic | 0.0 | 0.0 | 1.30x faster | Significant (t=76.90) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleIntFloatArithmetic | 0.0 | 0.0 | 1.37x faster | Significant (t=110.27) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleIntegerArithmetic | 0.0 | 0.0 | 1.37x faster | Significant (t=82.34) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleListComprehensions | 0.0 | 0.0 | 1.62x faster | Significant (t=122.24) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleListManipulation | 0.0 | 0.0 | 1.13x faster | Significant (t=53.62) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SimpleLongArithmetic | 0.0 | 0.0 | 1.23x faster | Significant (t=102.31) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SmallLists | 0.0 | 0.0 | 1.47x faster | Significant (t=57.67) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SmallTuples | 0.0 | 0.0 | 1.12x faster | Significant (t=50.66) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SpecialClassAttribute | 0.0 | 0.0 | 1.06x faster | Significant (t=8.98) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.SpecialInstanceAttribute | 0.0 | 0.0 | 1.12x faster | Significant (t=12.40) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.StringPredicates | 0.0 | 0.0 | 1.05x faster | Significant (t=22.45) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.StringSlicing | 0.0 | 0.0 | 1.09x faster | Significant (t=17.70) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.TryExcept | 0.0 | 0.0 | 1.43x faster | Significant (t=156.64) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.TryFinally | 0.0 | 0.0 | 1.05x faster | Significant (t=25.06) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.TryRaiseExcept | 0.0 | 0.0 | 1.04x slower | Significant (t=-21.91) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.TupleSlicing | 0.0 | 0.0 | 1.04x faster | Significant (t=5.06) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.UnicodePredicates | 0.0 | 0.0 | 1.06x faster | Significant (t=27.29) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.UnicodeProperties | 0.0 | 0.0 | 1.07x slower | Significant (t=-27.27) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.UnicodeSlicing | 0.0 | 0.0 | 1.08x faster | Significant (t=23.72) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.WithFinally | 0.0 | 0.0 | 1.20x faster | Significant (t=71.47) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| pybench.WithRaiseExcept | 0.0 | 0.0 | 1.08x faster | Significant (t=40.69) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| raytrace | 1.31 | 1.17 | 1.12x faster | Significant (t=23.57) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| regex_compile | 0.36 | 0.42 | 1.15x slower | Significant (t=-27.59) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| regex_effbot | 0.01 | 0.0 | 1.13x faster | Significant (t=9.91) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| regex_v8 | 0.05 | 0.04 | 1.03x faster | Significant (t=15.24) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| richards | 0.18 | 0.17 | 1.03x faster | Significant (t=3.79) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| spectral_norm | 0.32 | 0.29 | 1.08x faster | Significant (t=15.93) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| startup | 0.02 | 0.03 | 1.22x slower | Significant (t=-178.03) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| startup/no_site | 0.01 | 0.01 | 1.06x faster | Significant (t=28.82) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| telco | 0.02 | 0.01 | 1.18x faster | Significant (t=31.49) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
| unpack_sequence | 0.0 | 0.0 | 1.66x faster | Significant (t=69.05) |
+------------------------------------+---------------+---------------+--------------+-------------------------+
The following not significant results are hidden, use -v to show them:
call_simple, chameleon, fastpickle/pickle_dict, json/json_load, logging/formatted_output, logging/simple_output,
pidigits, pybench.ListSlicing, pybench.PythonFunctionCalls, pybench.PythonMethodCalls, pybench.Recursion,
pybench.StringMappings, pybench.UnicodeMappings, tornado_http.
@AtHeartEngineer
Copy link

The "change" column should be whatever the variable is -1. It isn't 1.07x faster, it is .07x faster. Just clarification

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment