Created
September 13, 2016 18:53
-
-
Save brettcannon/b57a86a41b4eb0ea64c3b837690c9662 to your computer and use it in GitHub Desktop.
Performance comparison between Python 3.5.2+ & 3.6b1+
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| Benchmark | 3.5-perf.json | 3.6-perf.json | Change | Significance | | |
+====================================+===============+===============+==============+=========================+ | |
| 2to3 | 0.68 | 0.62 | 1.09x faster | Significant (t=65.03) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| call_method | 0.02 | 0.02 | 1.07x faster | Significant (t=46.01) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| call_method_slots | 0.02 | 0.02 | 1.06x faster | Significant (t=29.45) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| call_method_unknown | 0.02 | 0.02 | 1.07x faster | Significant (t=39.01) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| chaos | 0.0 | 0.0 | 1.11x faster | Significant (t=32.86) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| django_template | 0.7 | 0.67 | 1.05x faster | Significant (t=18.01) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| elementtree/generate | 0.24 | 0.22 | 1.12x faster | Significant (t=37.47) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| elementtree/iterparse | 0.41 | 0.2 | 2.03x faster | Significant (t=245.94) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| elementtree/parse | 0.27 | 0.24 | 1.16x faster | Significant (t=32.72) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| elementtree/process | 0.2 | 0.17 | 1.13x faster | Significant (t=29.71) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| fannkuch | 1.25 | 0.98 | 1.28x faster | Significant (t=84.36) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| fastpickle/pickle | 0.0 | 0.0 | 1.08x faster | Significant (t=11.73) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| fastpickle/pickle_list | 0.0 | 0.0 | 1.05x faster | Significant (t=13.73) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| fastpickle/unpickle | 0.0 | 0.0 | 1.38x faster | Significant (t=22.84) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| fastpickle/unpickle_list | 0.0 | 0.0 | 2.26x faster | Significant (t=129.58) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| float | 0.3 | 0.28 | 1.07x faster | Significant (t=23.95) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| go | 0.54 | 0.53 | 1.02x faster | Significant (t=8.43) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| hexiom2/level_25 | 0.02 | 0.02 | 1.03x faster | Significant (t=14.82) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| json_dump_v2 | 0.03 | 0.02 | 1.15x faster | Significant (t=39.51) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| logging/no_output | 0.0 | 0.0 | 1.04x faster | Significant (t=8.31) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| mako | 0.04 | 0.04 | 1.06x slower | Significant (t=-22.74) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| meteor_contest | 0.21 | 0.19 | 1.07x faster | Significant (t=16.66) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| nbody | 0.26 | 0.23 | 1.12x faster | Significant (t=45.25) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| nqueens | 0.24 | 0.2 | 1.16x faster | Significant (t=43.43) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pathlib | 0.05 | 0.05 | 1.02x slower | Significant (t=-5.18) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.BuiltinFunctionCalls | 0.0 | 0.0 | 1.08x slower | Significant (t=-32.64) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.BuiltinMethodLookup | 0.0 | 0.0 | 1.23x faster | Significant (t=54.90) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareFloats | 0.0 | 0.0 | 1.20x faster | Significant (t=50.87) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareFloatsIntegers | 0.0 | 0.0 | 1.58x faster | Significant (t=80.25) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareIntegers | 0.0 | 0.0 | 1.24x faster | Significant (t=67.17) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareInternedStrings | 0.0 | 0.0 | 1.25x faster | Significant (t=36.71) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareLongs | 0.0 | 0.0 | 1.23x faster | Significant (t=78.35) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareStrings | 0.0 | 0.0 | 1.13x faster | Significant (t=44.88) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CompareUnicode | 0.0 | 0.0 | 1.13x faster | Significant (t=43.82) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.ComplexPythonFunctionCalls | 0.0 | 0.0 | 1.25x slower | Significant (t=-62.75) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.ConcatStrings | 0.0 | 0.0 | 1.07x faster | Significant (t=16.46) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.ConcatUnicode | 0.0 | 0.0 | 1.08x faster | Significant (t=19.27) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CreateInstances | 0.0 | 0.0 | 1.14x faster | Significant (t=29.49) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CreateNewInstances | 0.0 | 0.0 | 1.13x faster | Significant (t=26.90) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CreateStringsWithConcat | 0.0 | 0.0 | 1.16x faster | Significant (t=47.73) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.CreateUnicodeWithConcat | 0.0 | 0.0 | 1.16x faster | Significant (t=57.42) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.DictCreation | 0.0 | 0.0 | 1.93x faster | Significant (t=123.84) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.DictWithFloatKeys | 0.0 | 0.0 | 1.13x faster | Significant (t=40.70) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.DictWithIntegerKeys | 0.0 | 0.0 | 1.16x faster | Significant (t=36.43) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.DictWithStringKeys | 0.0 | 0.0 | 1.17x faster | Significant (t=21.18) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.ForLoops | 0.0 | 0.0 | 1.50x faster | Significant (t=137.07) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.IfThenElse | 0.0 | 0.0 | 1.48x faster | Significant (t=45.83) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.NestedForLoops | 0.0 | 0.0 | 1.28x faster | Significant (t=82.12) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.NestedListComprehensions | 0.0 | 0.0 | 1.54x faster | Significant (t=82.01) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.NormalClassAttribute | 0.0 | 0.0 | 1.06x faster | Significant (t=7.15) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.NormalInstanceAttribute | 0.0 | 0.0 | 1.10x faster | Significant (t=6.00) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SecondImport | 0.0 | 0.0 | 3.02x faster | Significant (t=269.54) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SecondPackageImport | 0.0 | 0.0 | 2.80x faster | Significant (t=226.09) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SecondSubmoduleImport | 0.0 | 0.0 | 2.96x faster | Significant (t=204.35) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleComplexArithmetic | 0.0 | 0.0 | 1.20x faster | Significant (t=48.87) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleDictManipulation | 0.0 | 0.0 | 1.03x slower | Significant (t=-10.36) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleFloatArithmetic | 0.0 | 0.0 | 1.30x faster | Significant (t=76.90) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleIntFloatArithmetic | 0.0 | 0.0 | 1.37x faster | Significant (t=110.27) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleIntegerArithmetic | 0.0 | 0.0 | 1.37x faster | Significant (t=82.34) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleListComprehensions | 0.0 | 0.0 | 1.62x faster | Significant (t=122.24) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleListManipulation | 0.0 | 0.0 | 1.13x faster | Significant (t=53.62) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SimpleLongArithmetic | 0.0 | 0.0 | 1.23x faster | Significant (t=102.31) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SmallLists | 0.0 | 0.0 | 1.47x faster | Significant (t=57.67) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SmallTuples | 0.0 | 0.0 | 1.12x faster | Significant (t=50.66) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SpecialClassAttribute | 0.0 | 0.0 | 1.06x faster | Significant (t=8.98) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.SpecialInstanceAttribute | 0.0 | 0.0 | 1.12x faster | Significant (t=12.40) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.StringPredicates | 0.0 | 0.0 | 1.05x faster | Significant (t=22.45) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.StringSlicing | 0.0 | 0.0 | 1.09x faster | Significant (t=17.70) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.TryExcept | 0.0 | 0.0 | 1.43x faster | Significant (t=156.64) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.TryFinally | 0.0 | 0.0 | 1.05x faster | Significant (t=25.06) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.TryRaiseExcept | 0.0 | 0.0 | 1.04x slower | Significant (t=-21.91) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.TupleSlicing | 0.0 | 0.0 | 1.04x faster | Significant (t=5.06) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.UnicodePredicates | 0.0 | 0.0 | 1.06x faster | Significant (t=27.29) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.UnicodeProperties | 0.0 | 0.0 | 1.07x slower | Significant (t=-27.27) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.UnicodeSlicing | 0.0 | 0.0 | 1.08x faster | Significant (t=23.72) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.WithFinally | 0.0 | 0.0 | 1.20x faster | Significant (t=71.47) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| pybench.WithRaiseExcept | 0.0 | 0.0 | 1.08x faster | Significant (t=40.69) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| raytrace | 1.31 | 1.17 | 1.12x faster | Significant (t=23.57) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| regex_compile | 0.36 | 0.42 | 1.15x slower | Significant (t=-27.59) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| regex_effbot | 0.01 | 0.0 | 1.13x faster | Significant (t=9.91) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| regex_v8 | 0.05 | 0.04 | 1.03x faster | Significant (t=15.24) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| richards | 0.18 | 0.17 | 1.03x faster | Significant (t=3.79) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| spectral_norm | 0.32 | 0.29 | 1.08x faster | Significant (t=15.93) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| startup | 0.02 | 0.03 | 1.22x slower | Significant (t=-178.03) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| startup/no_site | 0.01 | 0.01 | 1.06x faster | Significant (t=28.82) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| telco | 0.02 | 0.01 | 1.18x faster | Significant (t=31.49) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
| unpack_sequence | 0.0 | 0.0 | 1.66x faster | Significant (t=69.05) | | |
+------------------------------------+---------------+---------------+--------------+-------------------------+ | |
The following not significant results are hidden, use -v to show them: | |
call_simple, chameleon, fastpickle/pickle_dict, json/json_load, logging/formatted_output, logging/simple_output, | |
pidigits, pybench.ListSlicing, pybench.PythonFunctionCalls, pybench.PythonMethodCalls, pybench.Recursion, | |
pybench.StringMappings, pybench.UnicodeMappings, tornado_http. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
The "change" column should be whatever the variable is -1. It isn't 1.07x faster, it is .07x faster. Just clarification