Skip to content

Instantly share code, notes, and snippets.

@simonw

simonw/uv-help.md Secret

Last active November 8, 2024 14:17
Show Gist options
  • Save simonw/975dfa41e9b03bca2513a986d9aa3dcf to your computer and use it in GitHub Desktop.
Save simonw/975dfa41e9b03bca2513a986d9aa3dcf to your computer and use it in GitHub Desktop.
Download ZIP
Raw
uv-help.md

uv help needed with project test dependencies

I'm having trouble figuring out the correct way to use uv to replace my current use of pipenv.

See section at bottom, part of this was because of CONDA_PREFIX but I still have questions - also note that the Conda gotcha that caught me out here has been addressed in uv 0.5.

My current pipenv shell pattern

Here's the pattern I use. I start by creating a new project using a cookiecutter template:

cookiecutter gh:simonw/click-app
  [1/6] app_name (): demo-app
  [2/6] description (): Demo
  [3/6] hyphenated (demo-app): 
  [4/6] underscored (demo_app): 
  [5/6] github_username (): simonw
  [6/6] author_name (): Simon Willison

Now I have this:

find demo-app
demo-app
demo-app/LICENSE
demo-app/pyproject.toml
demo-app/tests
demo-app/tests/test_demo_app.py
demo-app/README.md
demo-app/.gitignore
demo-app/.github
demo-app/.github/workflows
demo-app/.github/workflows/publish.yml
demo-app/.github/workflows/test.yml
demo-app/demo_app
demo-app/demo_app/__init__.py
demo-app/demo_app/cli.py
demo-app/demo_app/__main__.py

I want to start hacking on demo-app using a dedicated virtual environment and pip install -e '.[test]' to start work. I currently do this:

cd demo-app
pipenv shell
# Now I'm in a fresh, activated virtual environment
pip install -e '.[test]'

That's installed my app itself in editable mode plus its test dependencies, which includes pytest. So now I can run:

python -m pytest

And see:

============================= test session starts ==============================
platform darwin -- Python 3.10.10, pytest-8.3.3, pluggy-1.5.0
rootdir: /private/tmp/demo-app
configfile: pyproject.toml
collected 1 item                                                               

tests/test_demo_app.py .                                                 [100%]

============================== 1 passed in 0.01s ===============================

I can also run my new demo-app directly, since it's been added to my pipenv environment as a command:

demo-app --help
Usage: demo-app [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here

And confirm the location of that demo-app executable with which:

which demo-app
/Users/simon/.local/share/virtualenvs/demo-app-AU_yVo5p/bin/demo-app

How would I do that with uv?

I can't figure out the right uv pattern for this. Ideally I'd like to avoid the "activate your virtual environment" step entirely and do everything with uv run.

Here's what I've tried that didn't work:

uvx cookiecutter gh:simonw/click-app

This works the same as above, I go through the same steps to create demo-app.

cd demo-app
uv run pytest

This doesn't work:

Using CPython 3.11.1
Creating virtual environment at: .venv
Installed 1 package in 2ms
============================= test session starts ==============================
platform darwin -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0
rootdir: /private/tmp/b/demo-app
configfile: pyproject.toml
plugins: icdiff-0.6, syrupy-4.5.0, asyncio-0.21.1, django-4.9.0, clarity-1.0.1, httpx-0.22.0, anyio-3.7.1, requests-mock-1.11.0, timeout-2.3.1, xdist-3.6.1
asyncio: mode=strict
collected 0 items / 1 error                                                    

==================================== ERRORS ====================================
___________________ ERROR collecting tests/test_demo_app.py ____________________
ImportError while importing test module '/private/tmp/b/demo-app/tests/test_demo_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_demo_app.py:2: in <module>
    from demo_app.cli import cli
E   ModuleNotFoundError: No module named 'demo_app'
=========================== short test summary info ============================
ERROR tests/test_demo_app.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.07s ===============================

Clearly I need to do the equivalent of pip install -e '.[test]' - but I can't figure out what that equivalent is!

For a moment I thought I had figured it out:

uv pip install -e '.[test]'

And now:

uv run pytest
================================================= test session starts ==================================================
platform darwin -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0
rootdir: /private/tmp/b/demo-app
configfile: pyproject.toml
plugins: icdiff-0.6, syrupy-4.5.0, asyncio-0.21.1, django-4.9.0, clarity-1.0.1, httpx-0.22.0, anyio-3.7.1, requests-mock-1.11.0, timeout-2.3.1, xdist-3.6.1
asyncio: mode=strict
collected 1 item                                                                                                       

tests/test_demo_app.py .                                                                                         [100%]

================================================== 1 passed in 0.01s ===================================================

But no... that's not right. I think it's a coincidence that that worked, because:

uv run which pytest
/opt/homebrew/Caskroom/miniconda/base/bin/pytest

How do I use pytest as a test dependency?

I don't want to run the globally installed pytest - I want to run a pytest that lives in my current virtual environment.

ls .venv/bin
activate
activate.bat
activate.csh
activate.fish
activate.nu
activate.ps1
activate_this.py
deactivate.bat
pydoc.bat
python
python3
python3.11

There's no pytest in there - so my uv pip install -e '.[test]' didn't install my test dependencies (from pyproject.toml) as expected.

I even tried this:

uv pip install pytest

But when I ran this I got an error:

uv run python -m pytest
/private/tmp/b/demo-app/.venv/bin/python3: No module named pytest

And even this returns no matches:

find .venv | grep pytest

So I seem to be missing something pretty fundamental here - how come uv pip install pytest didn't install pytest into the .venv directory there?

For the record, the pyproject.toml file in the project looks like this:

[project]
name = "demo-app"
version = "0.1"
description = "Demo"
readme = "README.md"
authors = [{name = "Simon Willison"}]
license = {text = "Apache-2.0"}
requires-python = ">=3.8"
classifiers = [
    "License :: OSI Approved :: Apache Software License"
]
dependencies = [
    "click"
]

[project.urls]
Homepage = "https://github.com/simonw/demo-app"
Changelog = "https://github.com/simonw/demo-app/releases"
Issues = "https://github.com/simonw/demo-app/issues"
CI = "https://github.com/simonw/demo-app/actions"

[project.scripts]
demo-app = "demo_app.cli:cli"

[project.optional-dependencies]
test = ["pytest"]

Also where is demo-app?

Since this is meant to be a CLI utility, I want to be able to do this:

uv run demo-app --help

To exercise the app as if it had been installed.

This appeared to work...

Usage: demo-app [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here

But then I ran this command:

uv run which demo-app

And got back:

/opt/homebrew/Caskroom/miniconda/base/bin/demo-app

I had intended to install demo-app in my current .venv/ folder managed by uv, but it looks like I accidentally installed it globally instead!

Wait, maybe it's because of CONDA_PREFIX

I ran this:

env | grep CONDA_PREFIX

And got:

CONDA_PREFIX=/opt/homebrew/Caskroom/miniconda/base

https://docs.astral.sh/uv/pip/environments/#discovery-of-python-environments says:

When running a command that mutates an environment such as uv pip sync or uv pip install, uv will search for a virtual environment in the following order:

  • An activated virtual environment based on the VIRTUAL_ENV environment variable.
  • An activated Conda environment based on the CONDA_PREFIX environment variable.
  • A virtual environment at .venv in the current directory, or in the nearest parent directory.

So that's probably why it's going wrong for me!

Trying this again:

cd /tmp
mkdir noc
cd noc
unset CONDA_PREFIX

uvx cookiecutter gh:simonw/click-app
# Go through the steps
cd demo
uv pip install -e '.[test]'
uv run pytest

That worked!

And I can run this:

uv run python -m demo
Usage: python -m demo [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here

There's just one remaining problem: the demo executable itself is nowhere to be found:

uv run demo
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/private/tmp/noc/demo/demo/__main__.py", line 1, in <module>
    from .cli import cli
ImportError: attempted relative import with no known parent package

And if I peek in .venv:

ls .venv/bin

I see no demo and no pytest either:

activate
activate.bat
activate.csh
activate.fish
activate.nu
activate.ps1
activate_this.py
deactivate.bat
pydoc.bat
python
python3
python3.11

So unsetting CONDA_PREFIX fixes some but not all of my problems.

I still can't figure out what commands to run such that the executables defined in my pyproject.toml or for things I have installed like pytest end up in the .venv/bin folder and can be run using uv run name-of-executable.

@simonw
Copy link
Author

simonw commented Nov 8, 2024

I just learned that uv 0.5 changed that Conda behavior in a way that directly addresses the point of confusion I suffered here:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment