Skip to content

Instantly share code, notes, and snippets.

@x-yuri
Last active February 14, 2021 12:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save x-yuri/348c9ca1a7a55344367e6b5379a1769f to your computer and use it in GitHub Desktop.
Save x-yuri/348c9ca1a7a55344367e6b5379a1769f to your computer and use it in GitHub Desktop.
#pip #python

pip installation options

First, to put things simply there're installation base and installtion schemes. Installation base provides a reference point, while installation schemes regulate the resulting directory layout (where things are put relative to the installation base).

pip uses distutils for figuring all that out. The latter defines 3 installation schemes:

The schemes are defined here. With the user scheme it's a bit more involved, some paths are relative to userbase, some to usersite. Those are roughly correspond to site.USER_BASE and site.USER_SITE. Which are set by getuserbase() and getusersitepackages(). site.getuserbase() defaults to ~/.local, site.getusersitepackages() to {userbase}/lib/python{py_version_short}/site-packages.

The default scheme is the prefix scheme:

...every time you run python setup.py install without any other options, you’re using it.

But you can change the prefix (installation base) with the --prefix option. The --user option selects the user scheme.

The --root options allows you to prepend to the resulting paths another prefix, be the path absolute or relative.

With --target it's more difficult. It installs files to a temporary directory using the home scheme, and then moves them to the target locations. It's like the home scheme, but {pure,plat}lib equals data. Some aspects about it though make me wonder if it's to be avoided. Like the warn_script_location flag. If True it makes it complain about scripts you're installing not being on PATH. With --target it's False. Also, if the package is already there it says:

WARNING: Target directory ... already exists. Specify --upgrade to force replacement.

Because with --target it doesn't check if the package is already installed somewhere.

Last but not least, the check_supported_wheels flag. Theoretically it enables you to install packages that are not supported by the current platform. Although I failed to find a way to reproduce this behavior. The best I could achieve is this:

$ docker run python:3.5-alpine pip install -t d \
    https://files.pythonhosted.org/packages/69/62/41a5a7ed8a474072d491521562ffbeb18a869c090c21506f890298433fab/factory_boy-3.2.0-py2.py3-none-any.whl
...
ERROR: Package 'factory-boy' requires a different Python: 3.5.10 not in '>=3.6'

(The message is triggered further down the road.) But there might be situations where it could let it slide.

But there's also the custom installation scheme. You can't pass options to distutils via command line, e.g.:

$ pip install \
    --install-option=--install-lib=./site-packages \
    --install-option=--install-scripts=./site-packages/bin \
    --install-option=--install-data=./site-packages \
    bottle
...
pip._internal.exceptions.CommandError: Location-changing options found in --install-option: ['--install-lib'] from command line. This is unsupported, use pip-level options like --user, --prefix, --root, and --target instead.

But you can put them into setup.cfg:

[install]
install-lib=./site-packages
install-scripts=./site-packages/bin
install-data=./site-packages

I'm going to give it a try for development in a docker container. To make the packages easily available from the host (the project root is bind mounted into the container), to not forget to specify the --target option.

And to give an example of what different layouts look like (relative to the installation base)...

Without options, with --prefix, or --user:

purelib: lib/python3.6/site-packages
platlib: lib/python3.6/site-packages
scripts: bin
data: .
headers: include/python3.6m/bottle

With --target:

purelib: .
platlib: .
scripts: bin
data: .
headers: include/python/bottle

With --root:

purelib: usr/local/lib/python3.6/site-packages
platlib: usr/local/lib/python3.6/site-packages
scripts: usr/local/bin
data: usr/local
headers: usr/local/include/python3.6m/bottle

Scripts are things like django-admin, data are desktop files, documentation (e.g. lorem-ipsum-generator).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment