Notes on Development with Python

PythonDevelopment.md

• Create Dev Virtual Environment
• Activtae Dev Virtual Environment
• Install Dev Requirements
• Run Tests
• Run Linter
• Build Sphinx Docs
• Publish Docs to GitHub Pages
• Build Package
• Upload Package to PyPi
• Install Package in Edit Mode

---

This guide assumes Python version 3.9+

Create Dev Virtual Environment

git clone https://github.com/<user>/<project>
cd <project>
python -m venv .venv_dev --upgrade-deps

Note: If using an IDE it will likely want it's own virtual environment. Make sure names don't collide. For IntelliJ be sure to assign it a different environment such as .venv_intellij.

Note: If you don't include the --upgrade-deps option you may end up with a virtual environment with older pip and setuptools. This can be problematic if you are trying to use a pyproject.toml file with an editable install, as that requires pip >= 21.3 and setuptools >= 64

Activate Dev Virtual Environment

Windows

.venv_dev\Scripts\activate.bat

Linux (SH Shell)

source .venv_dev/bin/activate

MacOS (CSH Shell)

source .venv_dev/bin/activate.csh

Install Dev Requirements

from Dev Virtual Environment:

pip install -r requirements_dev.txt

OR if a library without a requirements_dev.txt use:

pip install -e .[dev]

Note: You should only need to do this when you first setup your virtual environment.

Run Tests

from Dev Virtual Environment:

pytest

Note: Assumes you've got pytest in your requirements_dev.txt and installed.

Run Linter

from Dev Virtual Environment:

pylint src/<package-name>

Note: Assumes you've got pylint in your requirements_dev.txt and installed.

Build Sphinx Docs

from Dev Virtual Environment:

Windows

sphinx-build -b html docsrc/source build/docs

MacOS & Linux

python3 -m sphinx.cmd.build -b html docsrc/source build/docs

Publish Docs to GitHub Pages

• Checkout gh-pages branch of the project
• Update index.html to include new version
• Create a new directory named after the new version
• Dump generated docs into new directory
• Add/Commit/Push to GitHub
• Wait a minute or two for changes to show up

Build Package

from Dev Virtual Environment:

Windows

python -m build

MacOS and Linux

pip  install --upgrade pip wheel setuptools
python3 setup.py sdist bdist_wheel

Note: The latest versions of setuptools appear to automatically create a temporary virtual environment before building the package. It isn't clear to me yet the ramifications of this.

Upload Package to PyPi

from Dev Virtual Environment and after Build Package:

python -m twine upload --repository pypi dist/*

Note: Bump version in setup.cfg for a new release. The Sphinx docsrc/source/conf.py also has a version variable. You can release alpha/beta pre-releases that don't affect "latest" on pypi. Simply append "a0" on version string in setup.cfg to indicate alpha release 0. You can then increment to alpha release 1 with a1, and so on.

Note: Generally it's useful to "clean" before running the build, but sadly there doesn't appear to be a simple command for this that works cross platform. You can just manually delete the build and dist, _docsrc/source/autosummary directories.

Note: The JLab intercepting proxy breaks uploads to PyPi from inside the JLab intranet. Use --cert option. Proxy Notes

Note: To upload to PyPi you must have an account and generally you'll create an access token and store it in your home directory in a file named .pypirc

Install Package in Edit Mode

python -m pip install -e <package-name>

Note: This is useful to override the installed package in site-packages. It assumes you've got the source code, so you may need to git clone first.