pkg_resources.VersionConflict on pip after macOS upgrade

pip-version-conflict-after-macos-upgrade.md

pkg_resources.VersionConflict on pip after upgrade

Encountered this during the execution of an Ansible playbook after macOS has been upgraded to 10.5.4

Traceback (most recent call last):
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 584, in _build_master
    ws.require(__requires__)
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 901, in require
    needed = self.resolve(parse_requirements(requirements))
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 792, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pip 20.1.1 (/usr/local/lib/python3.7/site-packages), Requirement.parse('pip==20.0.2'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File \"/usr/local/Cellar/python/3.7.8/bin/pip3\", line 6, in <module>
    from pkg_resources import load_entry_point
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 3254, in <module>
    @_call_aside
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 3238, in _call_aside
    f(*args, **kwargs)
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 3267, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 586, in _build_master
    return cls._build_from_requirements(__requires__)
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 599, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File \"/usr/local/lib/python3.7/site-packages/pkg_resources/__init__.py\", line 787, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==20.0.2' distribution was not found and is required by the application

It is because the pip3 wrapper is importing a version of pip which doesn't exist anymore.

$ cat /usr/local/Cellar/python/3.7.8/bin/pip3

#!/usr/local/opt/python/bin/python3.7
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==20.0.2','console_scripts','pip3'
__requires__ = 'pip==20.0.2'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('pip==20.0.2', 'console_scripts', 'pip3')()
    )

To fix it, we need to modify this pip3 wrapper, to use the pip version mentioned in after pkg_resources.VersionConflict in the exception, in this case, it is 20.1.1

$ vi /usr/local/Cellar/python/3.7.8/bin/pip3

#!/usr/local/opt/python/bin/python3.7
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==20.1.1','console_scripts','pip3'
__requires__ = 'pip==20.1.1'
import re
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
    sys.exit(
        load_entry_point('pip==20.1.1', 'console_scripts', 'pip3')()
    )

How to get the path to the pip3 wrapper?

$ python3

Python 3.7.8 (default, Jul  4 2020, 10:17:17)
[Clang 11.0.3 (clang-1103.0.32.62)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python37.zip', '/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python3.7', '/usr/local/Cellar/python/3.7.8/Frameworks/Python.framework/Versions/3.7/lib/python3.7/lib-dynload', '/usr/local/lib/python3.7/site-packages']

/usr/local/Cellar/python/3.7.8/ is the path to Python.