python-development-essentials.md

Python Development Essentials

Managing multiple python versions

Important: Under windows the default python interpreter must be listed before all other python versions in the Path Environment Variable.

Running a specific python version

Example for python version 3.10 under Windows:

> py -3.10

Running pip for a specific python version

Example for python version 3.10 under Windows:

> py -3.10 -m pip

Upgrading Setuptools

> pip install --upgrade pip setuptools

conda

Package, dependency and environment management

Installation

For Windows:

Download miniconda and select add to path in the installer. After the installation run the following command and reopen the terminal:

> conda init powershell

Display conda information

> conda info

Updating conda

> conda update conda

List all virtual environments

> conda env list

Creating virtual environments

Creating environments for specifiy python versions

For environments that should be installed under the default location run:

> conda create -n <env name> python=<version>

and for environments that should be installed under a specific location run:

> conda create --prefix <location> python=<version>

Creating environments from env files

For environments that should be installed under the default location run:

> conda env create -f <env file>

and for environments that should be installed under a specific location run:

> conda env create --prefix <location> -f <env file>

Activating and deactivating environments

For environments installed under the default location run:

> conda activate <env name>

> conda deactivate <env name>

and for environments installed under a specific location run:

> conda activate <env location>

> conda deactivate <env location>

Automatic activation

If you want to automatically activate an environment under Windows when opening the terminal add the above activation command to your PowerShell profile.

Manging channels

Appending channels to the channel list

By default, conda prefers packages from a higher priority channel over any version from a lower priority channel. Therefore, you can now safely put channels at the bottom of your channel list to provide additional packages that are not in the default channels and still be confident that these channels will not override the core package set.

> conda config --append channels <channel name>

Installing packages

Searching for packages

Use the following command to search for packages in the default channels:

> conda search <package>

or visit anaconda.org to also search for packages in other channels.

Installing packages

> conda install <package>

List installed packages in current environment

> conda list

Updating packages

Updating a specific package:

> conda update <package>

Updating all packages:

> conda update --all

Updating python

To update micro versions (e.g. 3.10.0 => 3.10.1) run:

> conda update python

To update minor versions (e.g. 3.10.9 > 3.11.0) delete the environment with the python version you want to update and create a new environment from the env file which specifies the new python version.

Removing packages

> conda remove <package>

Removing unused packages and caches

> conda clean -all

Export or update environment.yml file for the current environment

Under Windows run:

> $envName = "<name>"; $output = ".\environment.yml"; `
  conda env export --from-history > $output; `
  (Get-Content $output) -Replace "^(name:).*$", "name: $envName" | Set-Content $output; `
  (conda env export).Where({ $_ -match "- pip:" }, "SkipUntil" ) | Add-Content $output; `
  (Get-Content $output) | Where-Object { $_ -notmatch "prefix" -and $_ -notmatch "autopep8" } | Set-Content $output

to get an basic env file that you can fine-tune by hand or do it completely manual by running:

Under Windows:

> conda env export > .\env1.yml; conda env export --from-history > .\env2.yml

Under Linux:

> conda env export > ./env1.yml && conda env export --from-history > ./env2.yml

and merging env1.yml and env2.yml to an environment.yml file.

Updating environments

Update the env file with the new packages and run for environments installed under the default location:

> conda env update -n <env name> --file <env file> --prune

and for environments installed under a specific location:

> conda env update --prefix <env location> --file <env file> --prune

--prune causes conda to remove any dependencies that are no longer required from the environment.

Restoring environments

List environment history:

> conda list --revisions

Restore environment to a previous revision:

> conda install --revision=<rev number>

Removing environments

For environments installed under the default location run:

> conda remove -n <env name> --all

and for environments installed under a specific location run:

> conda remove --prefix <location> --all

Opening conda environments under Windows in IntelliJ

Activate the desired environment in the terminal and run the following command to open the desired folder in IntelliJ:

> idea <path>

Opening the folder in this way ensures that all conda path variables are loaded correctly into IntelliJ.

conda Cheatsheet

For more commands take a look at the cheatsheet or the conda documentation.

venv

Tool for creating isolated virtual python environments.

Creating virtual environments

> python venv venv

To passthrough global installed packages to the virtual environment add the --system-site-packages flag and than if needed use pip install --ignore-installed to install newer versions of global installed packages in the virtual environmnent.

Creating environments for specific python versions

Example for python version 3.10 under Windows:

> py -3.10 -m venv venv

Activating environments

Under Windows run:

> .\venv\Scrips\Activate.ps1

Deactivating environments

> deactivate

pip-chill

Like pip freeze but lists only the packages that are not dependencies of installed packages.

Installation

> pip install pip-chill

Usage

Creating requirements files

> pip-chill > requirements.txt

To suppress version numbers use the --no-version flag.