Skip to content

Instantly share code, notes, and snippets.

@tchamberlin
Last active May 29, 2024 16:23
Show Gist options
  • Save tchamberlin/a4806d31556d3f1b27fd8ace7e2d7e34 to your computer and use it in GitHub Desktop.
Save tchamberlin/a4806d31556d3f1b27fd8ace7e2d7e34 to your computer and use it in GitHub Desktop.
How to manage Python and related tooling on Fedora 39

How to manage Python via pyenv and pipx on Fedora 39

You want to:

  • not break system Python
  • use multiple, modern versions of Python simultaneously
  • easily install applications (e.g. black, ruff, pylint) in isolated environments
  • keep applications up to date
  • easily upgrade Python

This guide covers two tools to help with the above:

  • pyenv: Install and manage multiple Python versions
  • pipx: Install and manage Python applications

Initial Install

Python

Python doesn't provide Linux binaries, so pyenv installs from source. It should work out of the box, but with some caveats:

  • python3-tkinter: If not installed, you will get warnings about lack of Tk support, and things like the matplotlib interactive plotter won't work
  • python3-xlib: ...honestly not sure why I installed this
sudo dnf install python3-tkinter python3-xlib

Now you are ready to move forward with installing pyenv

pyenv

Install pyenv itself, as well as the version of Python you want to use. I recommend specifying a Python version that is at least a few months old, to avoid issues with tooling availability. Here we install 3.11. This means that the latest 3.11.x will be installed (3.11.6 at time of writing)

Full pyenv installation instructions

curl https://pyenv.run | bash
pyenv install 3.11  # Use most recent Python version
pyenv global 3.11  # Default to most-recent 3.11 patch version

pipx

Full pipx installation instructions

pyenv shell 3.11  # Ensure that pipx is using the Python version you expect
python3 -m pip install --user pipx
python3 -m pipx ensurepath

# Print the bash code needed to enable shell completions
register-python-argcomplete pipx

# Manual: paste the output from above into ~/.bashrc

You can now install a tool, e.g. black:

pipx install black

Black is now globally available, neatly installed into its own virtual environment that you never even had to think about!

Upgrades

How do you maintain your installs going forward?

Scenario 1: New patch version of Python is released (e.g. 3.11.6)

Scenario: you have Python 3.11.5 installed, but 3.11.6 is released

pyenv has no support for "in place" upgrades, but it's fairly easy to work around:

# Update list of available versions
pyenv update

# Install latest patch version
pyenv install 3.11

# Optional: remove previous version
pyenv uninstall 3.11.5

Note: best practice is to upgrade your tooling to use the latest patch version as well. This is not required, but is a good idea.

By default, pipx installs each "tool" to its own venv, and each venv is tied to a specific Python version. Again, there is no "in-place" upgrade option, but it's easy to work around:

# Reinstall all venvs with new global Python version (3.11.6 in this ex.)
pipx reinstall-all --python="$(pyenv which python)"

You should see all of your applications re-installed with 3.11.6

Final note: eventually, your applications will start dropping support for whatever version of Python you installed them with, and you will be forced to do the above. But this is typically on the order of many years.

Scenario 2: New version of Python is released (e.g. 3.13.0)

This scenario differs from the above in that you probably don't immediately want to change your default Python to 3.13.0 -- wait a few months for tooling to catch up!

But, in order to have it available, you simply need to:

# Update list of available versions
pyenv update

# Install latest patch version
pyenv install 3.13

When you want to switch over and use it as the default, simply:

pyenv global 3.13

Then, eventually, follow the steps in the first scenario to port your pipx applications over to the new version. This doesn't need to happen immediately!

Scenario 3: Upgrade pipx applications

pipx makes it very easy to keep everything up to date:

pipx upgrade-all

That's it!

Things you should not do

Do not install packages directly into pyenv-managed Python installations

e.g. do not: $ pyenv exec pip install [package]

This is a very minor thing -- you aren't going to break anything; it's just a bit confusing. pyenv isn't tracking your package installs for you, so they will not carry over to new Python version installs. Why bother with the overhead of remembering to install packages each time; just let pipx handle it

Do not install "user" packages via the system's pip

e.g. do not: $ /usr/bin/python -m pip install --user [package]

You will find tons of examples that tell you to do this. That doesn't mean it's a good idea! Dependency conflicts between packages can result in breakages (because everything is being installed into a single Python env). pipx is a much better solution

DO NOT install system packages via pip

e.g. do not: $ sudo /usr/bin/python -m pip install [package]

This will break stuff! Maybe not immediately, but eventually you will regret doing this. You might not even be sure why things are broken. On fedora, use only dnf to manage system Python packages!

See:

Recommended Tools

Some applications that I actually use via pipx, to give an idea of what's out there. See also pipx's list

  • ruff: Replaces black, pylint, isort, and others
  • pre-commit: community-provided git pre-commit hooks
  • cookiecutter: "create projects swiftly"
  • tldr: "a collection of community-maintained help pages for command-line tools"
  • Project management:
    • hatch: "a modern, extensible Python project manager."
    • pdm: "a modern Python package and dependency manager supporting the latest PEP standards"
    • pip-tools: "a set of command line tools to help you keep your pip-based packages fresh, even when you’ve pinned them"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment