Porting code from Python 2 to Python 3 can be a daunting task. Tools like Futureize or Modernize can do most of the mechanical work for you, and Pylint can find obvious problems with code that's meant to be 2and3 compatible. You should absolutely be using these tools as they identify the lion's share of compatibility problems. Thanks to this work, it's really never been easier to port a large codebase to Python 3.
Even with these tools, however, porting code in a way that ensures identical behavior in Python 2 and Python 3 is tough. Python is a highly dynamic language and there is a huge breadth of changes between Python 2 and Python 3. Also, while we'd all love to work in code bases with 100% unit test coverage, the reality is unfortunately often very different. Given this, it's hard if not impossible for a static analysis tool to find potential all pitfalls, some of which might not cause unit tests to fail.
But there is hope! Buried deep in @brettcannon's excellent Porting Python 2 code to Python 3 document there's a brief reference to Python 2's -3 flag. The -3 flag will raise a DeprecationWarning at runtime if CPython encounters behavior that has either been removed or significantly changed in Python 3. You can use this in tests to flag potential problems, or even enable it in production to find potential problems with code that might not be well tested.
For example:
$ python -3
Python 2.7.12 (default, Jun 29 2016, 14:05:02)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> 3/2
__main__:1: DeprecationWarning: classic int division
1
>>>The above example is slightly contrived and caught by all of the static tools mentioned above. But, of course it's easy to come up with an example that will sail through these static tools:
$ python -3
Python 2.7.12 (default, Jun 29 2016, 14:05:02)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> number_value = 42
>>> number_value.__div__(12)
__main__:1: DeprecationWarning: classic int division
3
>>>Great! Python still identifies this code as problematic in Python 3. This helps highlight the real power of enabling checking at runtime. By having Python itself flag potential problems we get much higher assurance our code will work as is in Python 3.
As a concrete example, I've been using this during test runs, and it's helped to find some potential problems with very popular third party libraries like simplejson, mock, coveragepy, or pytest as well as my own code. Some of these issues were actually OK, but it was better to remove the incompatible behavior anyway to help clean up the output of the -3 flag.
Most commonly, I've been finding problems with classes declaring a __eq__ method but not a __hash__ method. In Python 2, if a class were to do this it would get object.__hash__ as it's default hash method, which is likely never what you actually want. For example:
class JustEq(object):
def __init__(self, x):
self.x = x
def __eq__(self, other):
return self.x == other.x
list_of_things = [JustEq(1), JustEq(2), JustEq(3)]
set_of_things = set(list_of_things) # Fails in Python 3
print(JustEq(1) in list_of_things) # Prints 'True'
print(JustEq(1) in set_of_things) # Prints 'False'. Wat.In Python 3, classes declaring __eq__ but not __hash__ get None as their __hash__ implementation and are thus unhashable. The above code will error out when converting list_of_things into a set. This means users of your code will see different behavior in Python 2 vs Python 3. I've since submitted a Pull Request to Pylint to flag this problem which should make it into their next release.
By default, these messages are just printed to stderr. If you're following the Twelve-Factor App methodology you can enable this flag in your production services and then you should then be able to track occurrences of these errors in your log-indexing service. These warnings can also be controlled via the warnings module or the -W command line flag to turn these warnings into runtime exceptions.
As noted above, there's a number of popular third party libraries that do not run cleanly under the -3 flag so you won't be able to use it everywhere (yet 😄). And while you can and should submit pull requests to fix those issues, to start getting value right now out of the -3 flag you can incrementally use the warnings module.
If you use pytest, you can trivially enable these warnings to fire during your test cases using a conftest.py file:
# conftest.py
import warnings
import sys
PY3K_WARNINGS = getattr(sys, 'py3kwarning', False)
if PY3K_WARNINGS:
def pytest_runtest_setup(item):
warnings.filterwarnings('error',
category=DeprecationWarning,
module='YOUR_MODULE_HERE')
def pytest_runtest_teardown(item):
warnings.filterwarnings('ignore', category=DeprecationWarning)Then, instead of invoking py.test on the command line, invoke Python with the pytest module:
python -3 -m pytest ...This will flag code with your code as it runs, but won't necessarily flag bad class declarations. Once you've got your code running cleanly, you can then remove the conftest.py file and instead use the -W flag:
python -3 -Werror::DeprecationWarning:YOUR_MODULE_HERE -m pytest ...A patch has been submitted to Python to enable this flag via an environment variable instead of as a command line flag. This will be hugely useful to enable these warnings when Python is spawned by another process (like Gunicorn or Pytest x-dist).
Python 3 is still happening. Python 2.8 isn't (or at least won't come from the core developers). The -3 flag gives developers a way to move past compatibility concerns by validating your code as it's running. Give it a try!
should work fine, havn't tested tho