Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
How to modularize your py.test fixtures

Using py.test is great and the support for test fixtures is pretty awesome. However, in order to share your fixtures across your entire module, py.test suggests you define all your fixtures within one single conftest.py file. This is impractical if you have a large quantity of fixtures -- for better organization and readibility, you would much rather define your fixtures across multiple, well-named files. But how do you do that? ...No one on the internet seemed to know.

Turns out, however, you can define fixtures in individual files like this:

tests/fixtures/add.py

import pytest

@pytest.fixture
def add(x, y):
    x + y

Then you can import these fixtures in your conftest.py:

tests/conftest.py

import pytest
from fixtures.add import add

...and then you're good to test!

tests/adding_test.py

import pytest

@pytest.mark.usefixtures("add")
def test_adding(add):
    assert add(2, 3) == 5

Because of the modularity, tests will have to be run with python -m py.test instead of py.test directly.

Copy link

ghost commented Sep 21, 2017

thanks

@AndrewBMartin
Copy link

I'm having trouble making this work when running multiple tests at the same time. Would appreciate any insight you might have. Please see the SO question.

Thank you

@boxysean
Copy link

thank you

@ikonst
Copy link

ikonst commented Mar 6, 2018

You can simply create "local pytest plugins" which can be nothing more than Python files with fixtures, e.g.

  • tests/unit/conftest.py:

    pytest_plugins = [
       "tests.unit.fixtures.some_stuff",
    ]
  • tests/unit/fixtures/some_stuff.py:

    import pytest
    
    @pytest.fixture
    def foo():
        return 'foobar'

@ryanjdillon
Copy link

ryanjdillon commented Oct 11, 2018

Awesome ikonst!

Also note that the respective directories referred to in tests.unit.fixtures.some_stuff" need to have __init__.py files for the plugins to be loaded by pytest.

https://docs.pytest.org/en/documentation-restructure/how-to/writing_plugins.html#requiring-loading-plugins-in-a-test-module-or-conftest-file

@kown7
Copy link

kown7 commented Apr 29, 2019

Fixtures with a

@pytest.fixture(scope="session", autouse=True)

scope need to be in the conftest.py file directly.

@jpadhye
Copy link

jpadhye commented May 8, 2019

With python3 you run it as python -m pytest

Otherwise you see following error:


$ python -m py.test
/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py:125: RuntimeWarning: 'py.test' found in sys.modules after import of package 'py', but prior to execution of 'py.test'; this may result in unpredictable behaviour
  warn(RuntimeWarning(msg))
/Users/jpadhye/.local/share/virtualenvs/sse-test-bench-4eGiHs-X/bin/python: loader for pytest cannot handle py.test

@Seneketh
Copy link

Seneketh commented Jan 9, 2020

Thanks for sharing!

@putnamhill
Copy link

Oh thank you! Just what I need at the moment.

@ramast
Copy link

ramast commented May 6, 2020

You can simply create "local pytest plugins" which can be nothing more than Python files with fixtures, e.g.

* tests/unit/conftest.py:
  ```python
  pytest_plugins = [
     "tests.unit.fixtures.some_stuff",
  ]
  ```

* tests/unit/fixtures/some_stuff.py:
  ```python
  import pytest
  
  @pytest.fixture
  def foo():
      return 'foobar'
  ```

I really like your idea!

I've replaced

       pytest_plugins = [
          "tests.unit.fixtures.some_stuff",
       ]

with

from glob import glob

pytest_plugins = [
   fixture.replace("/", ".").replace("fixtures.py", "fixtures") for fixture in glob("*/tests/fixtures.py")
]

Then any fixture located in */tests/fixtures.py will be added automatically without extra work from me. You can adjust the pattern to fit your application's structure.

@christian-steinmeyer
Copy link

christian-steinmeyer commented Sep 14, 2020

In case, someone else likes the combination of both ideas by @ramast and @ikonst, I've adapted it to look up all fixtures, defined in their own file under the fixtures folder:

# conftest.py

from glob import glob


def refactor(string: str) -> str:
    return string.replace("/", ".").replace("\\", ".").replace(".py", "")


pytest_plugins = [
    refactor(fixture) for fixture in glob("tests/fixtures/*.py") if "__" not in fixture
]

So, this should work, if you project setup looks like this:

./
├── src
├── tests
│   ├── conftest.py
│   ├── fixtures
│   │   ├── my_fixture.py

edit: I noticed, that I was executing pytest from tests instead of root. Fixed the discovery path and shortened string manipulation.

@ikonst
Copy link

ikonst commented Sep 14, 2020

I do something similar in my projects.

  • walk_packages.py:

    from pkgutil import ModuleInfo
    from pkgutil import walk_packages
    from types import ModuleType
    from typing import Iterable
    
    
    def get_packages_in_module(m: ModuleType) -> Iterable[ModuleInfo]:
        return walk_packages(m.__path__, prefix=m.__name__ + '.')  # type: ignore
    
    
    def get_package_paths_in_module(m: ModuleType) -> Iterable[str]:
        return [package.name for package in get_packages_in_module(m)]
  • conftest.py

    import tests.fixtures
    from tests.helpers.walk_packages import get_package_paths_in_module
    
    pytest_plugins = [
        ...
        *get_package_paths_in_module(tests.fixtures),
    ]

Admittedly the glob-and-string-substitution variant is a little shorter :)

@dgaikwad
Copy link

@peterhurford and @ikonst Thank you, this post helped me.

@espoirMur
Copy link

In case, someone else likes the combination of both ideas by @ramast and @ikonst, I've adapted it to look up all fixtures, defined in their own file under the fixtures folder:

# conftest.py

from glob import glob


def refactor(string: str) -> str:
    return string.replace("/", ".").replace("\\", ".").replace(".py", "")


pytest_plugins = [
    refactor(fixture) for fixture in glob("tests/fixtures/*.py") if "__" not in fixture
]

So, this should work, if you project setup looks like this:

./
├── src
├── tests
│   ├── conftest.py
│   ├── fixtures
│   │   ├── my_fixture.py

edit: I noticed, that I was executing pytest from tests instead of root. Fixed the discovery path and shortened string manipulation.

Thanks a lot man.. this work like a charm for me..

@RomanKhudobei
Copy link

RomanKhudobei commented Mar 25, 2021

You can simply create "local pytest plugins" which can be nothing more than Python files with fixtures, e.g.

* tests/unit/conftest.py:
  ```python
  pytest_plugins = [
     "tests.unit.fixtures.some_stuff",
  ]
  ```

* tests/unit/fixtures/some_stuff.py:
  ```python
  import pytest
  
  @pytest.fixture
  def foo():
      return 'foobar'
  ```

Thanks for you idea @ikonst!
Just made my conftest as package and add it as plugin!

@gauravkhuraana
Copy link

When i try to run the code

def add(x, y):

it tries to find x as fixtures.

Can we call a fixture with parameter? in above example add is a fixture method which has parameter

whenever we call fixture, we just mention fixture name,, so if i have to use add, where i can mention x or y..

@christian-steinmeyer
Copy link

Is this what you are looking for, @gauravkhuraana?

@gauravkhuraana
Copy link

@christian-steinmeyer thank you for replying.
I saw that parmeterization concept i see, that applies to test method instead of fixture

Here is my fixture

import pytest
@pytest.fixture
def sum2Nbrs(a,b):
    return a+b

The test function is like

@pytest.mark.usefixtures("sum2Nbrs")
def test_2NbrsAddViaFixture(sum2Nbrs):
    print("The sume of numbers is", sum2Nbrs(2,4))

When i run this function i get

below is the error

file S:\Automation\python\tests\test_fixturesCall.py, line 7
def test_2NbrsAddViaFixture(sum2Nbrs):
file S:\Automation\python\tests\fixtures\MathsFixtures.py, line 3
@pytest.fixture
def sum2Nbrs(a,b):

E fixture 'a' not found

i would like to call fixture and not test method with parameters as shown above.

My test function should take 2 parameters and give me the sum of them

@christian-steinmeyer
Copy link

How about

import pytest
@pytest.fixture
def sum2Nbrs():
    def sum(a, b):
        return a+b

    return sum

@gauravkhuraana
Copy link

Thanks a lot christian , this did what i wanted to achieve

Actual Code/ Fixtures

@pytest.fixture
def sum2Nbrs():
 def sum(a,b):
    return a+b
 return sum   

Calling Code/fixture

def test_2NbrsAddViaFixture(sum2Nbrs):
    print("The sume of numbers is", sum2Nbrs(2,4))

This is a good way to parameterize the fixtures by creating another function(sum) under them(sum2Nbrs).

Calling the function directly mentioning the fixture name without subfunction(sum2Nbrs(2,4) )
We did not explicit took the name sum while calling.

Thank you

@amitrokade47
Copy link

Does pytest try to import all the fixtures from the pytest_plugins variable? I was wondering if the method mentioned in this post can be applied to resolve dependency issues that occur with so many fixtures being present in a single file, something like only the fixtures that are needed are imported... (I cant necessarily have multiple conftest files as I want to import and resuse fixtures across different directories)?

@ikonst
Copy link

ikonst commented Oct 13, 2021

Does pytest try to import all the fixtures from the pytest_plugins variable?

Yes, every module in pytest_plugins is loaded and all fixtures in that module are made available to all tests.

@Ledorub
Copy link

Ledorub commented Nov 19, 2021

glob("tests/fixtures/*.py") if "__" not in fixture

Can be replaced with

glob("tests/fixtures/[!__]*.py")

or even

glob("tests/fixtures/[!_]*.py")

to also skip filenames that start with single underscore.

@eddiebergman
Copy link

Just a solution I've come across for anyone reading:

# conftest.py
here = os.path.dirname(os.path.realpath(__file__))

pytest_plugins = []

def _as_module(root: str, path: str) -> str:
    path = os.path.join(root, path)
    path = path.replace(here, "")
    path = path.replace(".py", "")
    path = path.replace(os.path.sep, ".")[1:]
    return "test." + path


for root, dirs, files in os.walk(here, topdown=True):
    dirs[:] = [d for d in dirs if d.startswith("test")]
    pytest_plugins += [_as_module(root, f) for f in files if f.endswith("fixtures.py")]

It allows you to have a fixtures file in any subdirectory

.../test
   __init__.py
   conftest.py
   /test_f1
      __init__.py
      fixtures.py
   
      /test_subf1
         __init__.py
         fixtures.py
   
      /test_subf2
         __init__.py
         endswith_fixtures.py
         extra_fixtures.py
   
   /test_f2
      __init__.py
      fixtures.py

@ikonst
Copy link

ikonst commented Jan 11, 2022

@eddiebergman That's what the code in https://gist.github.com/peterhurford/09f7dcda0ab04b95c026c60fa49c2a68#gistcomment-3453453 (using pkgutil.walk_packages) already does.

@eddiebergman
Copy link

@ikonst Ahh, I didn't know it supported the same layout with fixtures in each dir, very nice

@joe-agent
Copy link

You can simply create "local pytest plugins" which can be nothing more than Python files with fixtures, e.g.

* tests/unit/conftest.py:
  ```python
  pytest_plugins = [
     "tests.unit.fixtures.some_stuff",
  ]
  • tests/unit/fixtures/some_stuff.py:
    import pytest
    
    @pytest.fixture
    def foo():
        return 'foobar'

Thanks for you idea @ikonst! Just made my conftest as package and add it as plugin!

Can you share how did you share the conftest as a package? Can that package be pip installed across other repo/projects?

@ikonst
Copy link

ikonst commented Apr 14, 2022

There's no much difference between a conftest.py and a pytest plugin (only one I can think of, is that a top-level conftest can have a pytest_plugins section).

If you want to make a Python package with pytest fixtures, see https://docs.pytest.org/en/6.2.x/writing_plugins.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment