uv-help.md

uv help needed with project test dependencies

I'm having trouble figuring out the correct way to use uv to replace my current use of pipenv.

See section at bottom, part of this was because of CONDA_PREFIX but I still have questions - also note that the Conda gotcha that caught me out here has been addressed in uv 0.5.

My current pipenv shell pattern

Here's the pattern I use. I start by creating a new project using a cookiecutter template:

cookiecutter gh:simonw/click-app

  [1/6] app_name (): demo-app
  [2/6] description (): Demo
  [3/6] hyphenated (demo-app): 
  [4/6] underscored (demo_app): 
  [5/6] github_username (): simonw
  [6/6] author_name (): Simon Willison

Now I have this:

find demo-app

demo-app
demo-app/LICENSE
demo-app/pyproject.toml
demo-app/tests
demo-app/tests/test_demo_app.py
demo-app/README.md
demo-app/.gitignore
demo-app/.github
demo-app/.github/workflows
demo-app/.github/workflows/publish.yml
demo-app/.github/workflows/test.yml
demo-app/demo_app
demo-app/demo_app/__init__.py
demo-app/demo_app/cli.py
demo-app/demo_app/__main__.py

I want to start hacking on demo-app using a dedicated virtual environment and pip install -e '.[test]' to start work. I currently do this:

cd demo-app
pipenv shell
# Now I'm in a fresh, activated virtual environment
pip install -e '.[test]'

That's installed my app itself in editable mode plus its test dependencies, which includes pytest. So now I can run:

python -m pytest

And see:

============================= test session starts ==============================
platform darwin -- Python 3.10.10, pytest-8.3.3, pluggy-1.5.0
rootdir: /private/tmp/demo-app
configfile: pyproject.toml
collected 1 item                                                               

tests/test_demo_app.py .                                                 [100%]

============================== 1 passed in 0.01s ===============================

I can also run my new demo-app directly, since it's been added to my pipenv environment as a command:

demo-app --help

Usage: demo-app [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here

And confirm the location of that demo-app executable with which:

which demo-app

/Users/simon/.local/share/virtualenvs/demo-app-AU_yVo5p/bin/demo-app

How would I do that with uv?

I can't figure out the right uv pattern for this. Ideally I'd like to avoid the "activate your virtual environment" step entirely and do everything with uv run.

Here's what I've tried that didn't work:

uvx cookiecutter gh:simonw/click-app

This works the same as above, I go through the same steps to create demo-app.

cd demo-app
uv run pytest

This doesn't work:

Using CPython 3.11.1
Creating virtual environment at: .venv
Installed 1 package in 2ms
============================= test session starts ==============================
platform darwin -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0
rootdir: /private/tmp/b/demo-app
configfile: pyproject.toml
plugins: icdiff-0.6, syrupy-4.5.0, asyncio-0.21.1, django-4.9.0, clarity-1.0.1, httpx-0.22.0, anyio-3.7.1, requests-mock-1.11.0, timeout-2.3.1, xdist-3.6.1
asyncio: mode=strict
collected 0 items / 1 error                                                    

==================================== ERRORS ====================================
___________________ ERROR collecting tests/test_demo_app.py ____________________
ImportError while importing test module '/private/tmp/b/demo-app/tests/test_demo_app.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/opt/homebrew/Caskroom/miniconda/base/lib/python3.10/importlib/__init__.py:126: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_demo_app.py:2: in <module>
    from demo_app.cli import cli
E   ModuleNotFoundError: No module named 'demo_app'
=========================== short test summary info ============================
ERROR tests/test_demo_app.py
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.07s ===============================

Clearly I need to do the equivalent of pip install -e '.[test]' - but I can't figure out what that equivalent is!

For a moment I thought I had figured it out:

uv pip install -e '.[test]'

And now:

uv run pytest

================================================= test session starts ==================================================
platform darwin -- Python 3.10.10, pytest-8.3.2, pluggy-1.5.0
rootdir: /private/tmp/b/demo-app
configfile: pyproject.toml
plugins: icdiff-0.6, syrupy-4.5.0, asyncio-0.21.1, django-4.9.0, clarity-1.0.1, httpx-0.22.0, anyio-3.7.1, requests-mock-1.11.0, timeout-2.3.1, xdist-3.6.1
asyncio: mode=strict
collected 1 item                                                                                                       

tests/test_demo_app.py .                                                                                         [100%]

================================================== 1 passed in 0.01s ===================================================

But no... that's not right. I think it's a coincidence that that worked, because:

uv run which pytest

/opt/homebrew/Caskroom/miniconda/base/bin/pytest

How do I use pytest as a test dependency?

I don't want to run the globally installed pytest - I want to run a pytest that lives in my current virtual environment.

ls .venv/bin

activate
activate.bat
activate.csh
activate.fish
activate.nu
activate.ps1
activate_this.py
deactivate.bat
pydoc.bat
python
python3
python3.11

There's no pytest in there - so my uv pip install -e '.[test]' didn't install my test dependencies (from pyproject.toml) as expected.

I even tried this:

uv pip install pytest

But when I ran this I got an error:

uv run python -m pytest

/private/tmp/b/demo-app/.venv/bin/python3: No module named pytest

And even this returns no matches:

find .venv | grep pytest

So I seem to be missing something pretty fundamental here - how come uv pip install pytest didn't install pytest into the .venv directory there?

For the record, the pyproject.toml file in the project looks like this:

[project]
name = "demo-app"
version = "0.1"
description = "Demo"
readme = "README.md"
authors = [{name = "Simon Willison"}]
license = {text = "Apache-2.0"}
requires-python = ">=3.8"
classifiers = [
    "License :: OSI Approved :: Apache Software License"
]
dependencies = [
    "click"
]

[project.urls]
Homepage = "https://github.com/simonw/demo-app"
Changelog = "https://github.com/simonw/demo-app/releases"
Issues = "https://github.com/simonw/demo-app/issues"
CI = "https://github.com/simonw/demo-app/actions"

[project.scripts]
demo-app = "demo_app.cli:cli"

[project.optional-dependencies]
test = ["pytest"]

Also where is demo-app?

Since this is meant to be a CLI utility, I want to be able to do this:

uv run demo-app --help

To exercise the app as if it had been installed.

This appeared to work...

Usage: demo-app [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here

But then I ran this command:

uv run which demo-app

And got back:

/opt/homebrew/Caskroom/miniconda/base/bin/demo-app

I had intended to install demo-app in my current .venv/ folder managed by uv, but it looks like I accidentally installed it globally instead!

Wait, maybe it's because of CONDA_PREFIX

I ran this:

env | grep CONDA_PREFIX

And got:

CONDA_PREFIX=/opt/homebrew/Caskroom/miniconda/base

https://docs.astral.sh/uv/pip/environments/#discovery-of-python-environments says:

When running a command that mutates an environment such as uv pip sync or uv pip install, uv will search for a virtual environment in the following order:

• An activated virtual environment based on the VIRTUAL_ENV environment variable.
• An activated Conda environment based on the CONDA_PREFIX environment variable.
• A virtual environment at .venv in the current directory, or in the nearest parent directory.

So that's probably why it's going wrong for me!

Trying this again:

cd /tmp
mkdir noc
cd noc
unset CONDA_PREFIX

uvx cookiecutter gh:simonw/click-app
# Go through the steps
cd demo
uv pip install -e '.[test]'
uv run pytest

That worked!

And I can run this:

uv run python -m demo

Usage: python -m demo [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here

There's just one remaining problem: the demo executable itself is nowhere to be found:

uv run demo

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/private/tmp/noc/demo/demo/__main__.py", line 1, in <module>
    from .cli import cli
ImportError: attempted relative import with no known parent package

And if I peek in .venv:

ls .venv/bin

I see no demo and no pytest either:

activate
activate.bat
activate.csh
activate.fish
activate.nu
activate.ps1
activate_this.py
deactivate.bat
pydoc.bat
python
python3
python3.11

So unsetting CONDA_PREFIX fixes some but not all of my problems.

I still can't figure out what commands to run such that the executables defined in my pyproject.toml or for things I have installed like pytest end up in the .venv/bin folder and can be run using uv run name-of-executable.

From Charlie Marsh:

First: uv run doesn't install extras by default, but you have pytest in an optional group.

There are a few ways to solve this:

1. uv run --extra test pytest will include the test group. (You can omit --extra test from subsequent uv run commands; we won't "remove" it.)

2. You can run uv sync --extra test before uv run pytest, to add the extra to the environment. Again, we won't remove the extra once you've installed it.

3. You can make it a dev dependency instead: https://docs.astral.sh/uv/concepts/dependencies/#development-dependencies. We do sync dev dependencies by default, so this would "just work". (These use non-standard metadata, but it just got standardized in PEP 735 and we'll release support for the standardized version this week.)

Second: Why is demo-app missing?

The answer here is specific to uv. We support two kinds of projects: packages and non-packages. You want the former in this case, because you actually want to install the package in the environment. (We used to require this, but a lot of people want to be able to create lightweight projects that are just collections of scripts and don't need to be buildable / installable into an environment. We call those non-package projects.)

We consider a project to be a "package" if [build-system] is defined or you set tool.uv.package = true

Otherwise, we don't install the project itself into the environment.

So you just need to make the build-system explicit by adding:

[build-system]
requires = ["setuptools>=42"]
build-backend = "setuptools.build_meta"

In short, if you add this to your pyproject.toml, then uv run pytest works as expected:

[tool.uv]
dev-dependencies = ["pytest"]
package = true

But I've outlined other options above (use uv run --extra test pytest instead of dev-dependencies; set an explicit build-system instead of package = true) for completeness.

The solution I like is to add that [build-system] block to my template, and then tell people to run uv sync --extra test the first time they checkout the repo.

Made that change to the template: simonw/click-app@2ed1748

It works:

% uvx cookiecutter gh:simonw/click-app
You've downloaded /Users/simon/.cookiecutters/click-app before. Is it okay to 
delete and re-download it? [y/n] (y): y
  [1/6] app_name (): demo-app
  [2/6] description (): Demo
  [3/6] hyphenated (demo-app): 
  [4/6] underscored (demo_app): 
  [5/6] github_username (): simonw
  [6/6] author_name (): Simon Willison
% cd demo-app 
% uv sync --extra test
Using CPython 3.11.1
Creating virtual environment at: .venv
Resolved 9 packages in 174ms
   Built demo-app @ file:///private/tmp/ucli/demo-app
Prepared 1 package in 587ms
Installed 6 packages in 7ms
 + click==8.1.7
 + demo-app==0.1 (from file:///private/tmp/ucli/demo-app)
 + iniconfig==2.0.0
 + packaging==24.1
 + pluggy==1.5.0
 + pytest==8.3.3
% uv run pytest
============================= test session starts ==============================
platform darwin -- Python 3.11.1, pytest-8.3.3, pluggy-1.5.0
rootdir: /private/tmp/ucli/demo-app
configfile: pyproject.toml
collected 1 item                                                               

tests/test_demo_app.py .                                                 [100%]

============================== 1 passed in 0.03s ===============================
% uv run demo-app
Usage: demo-app [OPTIONS] COMMAND [ARGS]...

  Demo

Options:
  --version  Show the version and exit.
  --help     Show this message and exit.

Commands:
  command  Command description goes here
% uv run which demo-app
/private/tmp/ucli/demo-app/.venv/bin/demo-app

I just learned that uv 0.5 changed that Conda behavior in a way that directly addresses the point of confusion I suffered here:

• astral-sh/uv#8940 (comment)