Skip to content

Instantly share code, notes, and snippets.

@hashar
Created September 19, 2023 04:15
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save hashar/123df97f1a4ec0fec6f50ecb789d1a45 to your computer and use it in GitHub Desktop.
Save hashar/123df97f1a4ec0fec6f50ecb789d1a45 to your computer and use it in GitHub Desktop.
pip requirements and constraints

Given a list of unpinned dependencies:

name=requirements.txt 
mwparserfromhell
pyyaml
pywikibot

Resolve the dependencies (ideally in a container with a controlled version of python):

$ virtualenv demo
$ . demo/bin/activate
(demo)$ pip install -r requirements.txt
...
Installing collected packages: urllib3, pyyaml, mwparserfromhell, idna, charset-normalizer, certifi, requests, pywikibot
Successfully installed certifi-2023.7.22 charset-normalizer-3.2.0 idna-3.4 mwparserfromhell-0.6.5 pywikibot-8.3.2 pyyaml-6.0.1 requests-2.31.0 urllib3-2.0.4

Generate the list of installed modules which includes transitive dependencies and capture it in a constraints.txt file:

(demo)$ pip freeze | tee constraints.txt
certifi==2023.7.22
charset-normalizer==3.2.0
idna==3.4
mwparserfromhell==0.6.5
pywikibot==8.3.2
PyYAML==6.0.1
requests==2.31.0
urllib3==2.0.4

And later you can install them using pip3 install -r requirements.txt -c constraints.txt.

# uninstall everything from the virtualenv
(demo)$ pip uninstall --yes $(pip freeze)
...
(demo)$ pip freeze
(demo)$ pip3 install -r requirements.txt -c constraints.txt
Installing collected packages: urllib3, pyyaml, mwparserfromhell, idna, charset-normalizer, certifi, requests, pywikibot
Successfully installed certifi-2023.7.22 charset-normalizer-3.2.0 idna-3.4 mwparserfromhell-0.6.5 pywikibot-8.3.2 pyyaml-6.0.1 requests-2.31.0 urllib3-2.0.4

(demo)$ pip3 install -r requirements.txt -c constraints.txt
Requirement already satisfied: mwparserfromhell in ./lib/python3.11/site-packages (from -r requirements.txt (line 1)) (0.6.5)
Requirement already satisfied: pyyaml in ./lib/python3.11/site-packages (from -r requirements.txt (line 2)) (6.0.1)
Requirement already satisfied: pywikibot in ./lib/python3.11/site-packages (from -r requirements.txt (line 3)) (8.3.2)
Requirement already satisfied: setuptools>=48.0.0 in ./lib/python3.11/site-packages (from pywikibot->-r requirements.txt (line 3)) (68.1.2)
Requirement already satisfied: requests>=2.21.0 in ./lib/python3.11/site-packages (from pywikibot->-r requirements.txt (line 3)) (2.31.0)
Requirement already satisfied: charset-normalizer<4,>=2 in ./lib/python3.11/site-packages (from requests>=2.21.0->pywikibot->-r requirements.txt (line 3)) (3.2.0)
Requirement already satisfied: idna<4,>=2.5 in ./lib/python3.11/site-packages (from requests>=2.21.0->pywikibot->-r requirements.txt (line 3)) (3.4)
Requirement already satisfied: urllib3<3,>=1.21.1 in ./lib/python3.11/site-packages (from requests>=2.21.0->pywikibot->-r requirements.txt (line 3)) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in ./lib/python3.11/site-packages (from requests>=2.21.0->pywikibot->-r requirements.txt (line 3)) (2023.7.22)

It is not a lock file though cause those dependencies versions can well be changed somehow. To lock them, one needs to add a checksum for each dependencies but pip does not provide out of the box tooling to do so.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment