Keywords, tags: python packaging setup.py setuptools
Purpose: describe how to make a "package" of a Python program for other people to use. This introduction builds on the excellent oa-packaging-guide-preview.readthedocs.io — read that first.
|-- my_package
| |-- my_package.py
| |-- __init__.py
| `-- test
| `-- test_my_package.py
|-- README.md
"Setup files" that describe the package:
|-- setup.py
|-- setup.cfg
|-- MANIFEST.in
|-- pyproject.toml
|-- changelog.txt
|-- LICENSE.BSD
These files are packaged, put into big single files for PyPI, by
python setup.py bdist_wheel # make a .whl binary distribution
python setup.py sdist # make a .tar.gz source distribution
(What's the difference between bdists and sdists ?
For python, both will be mostly .py text files;
sdists have additional files listed in MANIFEST.in.
For c programs, a bdist may be mostly binary .o and .dylib files,
an sdist .c .cpp .h .
An analogy:
- wheel: call-a-pizza, ready to eat
- sdist: a box of raw ingredients, with a complicated recipe.)
A "wheel" is a single file my_package-...whl that contains my_package/... files
plus files needed for pip install, as shown above.
There may be different wheels for python 2 / python3, macOS / Linux / Windows, etc.
Wheels are in .zip format, with suffix .whl instead of .zip;
unzip -l my_package-...whl lists all the files inside one.
First define a couple of shell variable to use in the following commands:
export PKG=my_package
export USERSITE=~/Library/Python/3.7
# where "pip install --user" puts packages on macOS
# for other systems see pip install -h
1: Make a wheel:
python setup.py bdist_wheel | tee 24feb-bdist_wheel.log
# output: e.g. dist/$PKG-version-py3-none-any.whl
egrep -i 'warning|error' 24feb-bdist_wheel.log # check the log
unzip -l dist/*.whl # list the files inside
2: Install it in the directory $USERSITE, with pip:
pip install --user dist/*.whl
# or --prefix $dir
tree $USERSITE/$PKG* # if you have "tree"
ls -RF $USERSITE/$PKG* # if not
3: Test the install:
cd to another directory
python or IPython
import package
print( package.__path__, package.__version__ )
cd $USERSITE/$PKG/test; pytest *.py
You could mail this dist/*.whl file to a friend to try it out:
pip install --user --no-deps $PKG*.whl, see pip install -h.
A source distribution of a Python package is similar to a wheel — for simple packages. Building from source, though, may need to compile C or C++ or Fortran, and link to the right versions of 14 other packages and libraries; lots of things can go wrong.
1: Make an sdist:
python setup.py sdist | tee 24feb-sdist.log
# output: dist/$PKG-version.tar.gz
egrep -i 'warning|error' 24feb-sdist.log # check the log
tar -tv -f dist/*.tar.gz # list the files inside
# sdists include the files listed in MANIFEST.in, bdist_wheels don't
2: Install and test it locally, with pip:
pip install --user dist/*.tar.gz
# Test the install as for a wheel
To make both together:
python setup.py bdist_wheel sdist | tee 24feb-wheel-sdist.log
# egrep the log, list dist/*, install locally, test the install, as above.
To upload dist/*.whl and dist/*.tar.gz to PyPi for anyone to pip install,
see twine.
(I had trouble registering on pypi.org,
got "400 Bad CSRF Token" on macOS 10.10, Firefox 70.0.1;
Safari worked.)
First twine upload to test.pypi.org,
then test both the wheel and the source dist from there:
# pip uninstall --user $PKG
pip install --user --only-binary :all: [--no-deps] $PKG # wheel
# import $PKG
pip install --user --no-binary :all: [--no-deps] $PKG # sdist
Then follow the last half of
https://packaging.python.org/tutorials/packaging-projects
(the first half uses setup.py only, not setup.cfg and MANIFEST.in).
Where did pip install put the files on disk ?
echo n | pip uninstall package
# /.../python3.7/site-packages/$PKG/*
# /.../python3.7/site-packages/$PKG-2020.2.20.dist-info/*
# n: don't remove (must be a better way)
The version string, e.g. "0.1.2",
is in two files here, both setup.py and $PKG/__init__.py.
Single-sourcing-package-version
lists 7 ways ! of putting it in one —
for simple packages MTTIW, more trouble than it's worth.
For bigger / more professional projects, see virtual environments and pipenv.
https://docs.python.org/3.7/installing/index.html
https://python-packaging-tutorial.readthedocs.io
https://packaging.python.org/tutorials/packaging-projects/
(uses setup.py only, not setup.cfg and MANIFEST.in)
Warning: some of the python packaging info on the web is, in March 2020, outdated or misleading. Most unfortunately, the python packaging community seems to have agreed to disagree on similar-but-different files and programs.
Whenever something can be done in two ways, someone will be confused.
— B. Stroustrup
Cheers
— denis-bz-py at t-online dot de
3 March, 27 July 2020