Python3 Upgrade Notes

notes.md

General

• Don't fork to a new branch, it will be impossible to merge back
• If only a handful of developers work on the branch, nobody else is comfortable writing python3 when you're ready to go
• You need to have a two-language compatible codebase

Stages

1. Third-party compatibility first
   • No new dependencies without Py3 support
   • Figure out which things just exist in Py3 now and update dependency expectation
   • Modernize CLI tool for updating
     • Do one type of change across all files instead of all of the necessary changes for one file at a time
2. Unit-tests eventually passing
   • You won't be able to prevent new modules/code from breaking the existing work until the tests are passing
   • Next while should be spent on getting all of the tests passing
   • Use an inclusion list for python3 mode when running tests at first
     • Anything in the inclusion list must pass
   • Eventually the inclusion list becomes an exclusion list because the inclusion list is too long
     • Everything that isn't in the exclusion list must pass
3. Production QA testing and final rollout
   • Running both versions in parallel
     • Initially only developers are doing QA on the python3 instances
     • After you become comfortable with developers using the py3 sandbox, advise CS and the rest of the company to use the py3 instances while our users are still using py2 instances
       • Finally switch the users over to the py3 instances and retire the py2 instances

Important things to consider

Helper functions

ensure_binary/ensure_str/ensure_text

Wrap all string usage and validate the contents

Iterators

map/filter/etc are now iterators/generators that will only evaluate once. You can wrap them in list to get equivalent behaviour to py2.

Strings

b'' will be the most significant change to your codebase. With Instagram the addition of this letter to strings accounted for a 12% storage capacity increase for their code.