py.test session fixtures

README.md

Pytest's session fixture scope is really handy, but there are some things which need to execute only once per actual test session. For example, one of my session fixtures sets up DB tables at the start of the test session and tears them down at the end. (I use PostgreSQL's nested transactions to keep from having to drop and recreate tables between each individual test.)

@pytest.fixture(scope='session')
def dbtables(request, sqlengine):
    Base.metadata.create_all(sqlengine)

    def teardown():
        Base.metadata.drop_all(sqlengine)

    request.addfinalizer(teardown)

This works great when tests are run sequentially, but when you try to run them in parallel using the pytest-xdist plugin (py.test -n4), each test runner individually tries to create and drop the tables, which simply doesn't work well.

I'm informed an alternative that does work is to use the pytest_sessionstart plugin function, but unfortunately this means that the fixtures upon which the dbtables depend will also have to be in the sessionstart plugin function, and that gets ugly very quickly.

I propose a new scope level provided by pytest-xdist, which provides a session-level scope which executes only once for all test runners, perhaps called 'xdist-session'. When one test runner finds an xdist-session fixture is required, it notifies the master, which then picks one of the slaves to execute the fixture. (The result of the fixture, if any, would need to be execnet-pickleable.) Once the fixture is executed, the master notifies the slaves and they resume running the tests. Likewise, at the end of the test session, the master waits for all the slaves to have finished before instructing the slave to execute any finalizers attached.

If the slave dies before executing the finalizers, they wouldn't be executed in this model. This limitation could be mitigated by having the master set up one slave to do nothing but execute xdist-session fixtures; since it won't execute tests, it is less likely to crash.

Of course, a test suite is not always run in parallel. Many xdist-session fixtures would easily fall back to running in session scope, but some test suites may need to have two similar but distinct fixtures, so there may need to be a way to have a 'guard' on a fixture so that it isn't used depending on the test invocation. I'm not sure how best to implement this.

conftest.py

import os

import pytest

from sqlalchemy import engine_from_config

from pyramid.paster import get_appsettings

from ..models import (
    DBSession,
    Base,
    )

@pytest.fixture(scope='session')
def appsettings(request):
    config_uri = os.path.abspath(request.config.option.ini)
    return get_appsettings(config_uri)

@pytest.fixture(scope='session')
def sqlengine(request, appsettings):
    engine = engine_from_config(appsettings, 'sqlalchemy.')
    return engine

@pytest.fixture(scope='session')
def dbtables(request, sqlengine):
    Base.metadata.create_all(sqlengine)

    def teardown():
        Base.metadata.drop_all(sqlengine)

    request.addfinalizer(teardown)

@pytest.fixture()
def dbtransaction(request, dbtables, sqlengine):
    connection = sqlengine.connect()
    transaction = connection.begin()
    DBSession.configure(bind=connection)
 
    def teardown():
        transaction.rollback()
        connection.close()
        DBSession.remove()
 
    request.addfinalizer(teardown)
 
    return connection

def pytest_addoption(parser):
    parser.addoption("--ini", action="store", metavar="INI_FILE", help="use INI_FILE to configure SQLAlchemy")

xdist-session is interesting. I have fixtures which may provide different results and get occasional assertions error because all processes don't share the same view. Did you get a chance to implement it? Or even an issue to follow?

You'll want to look at https://bitbucket.org/hpk42/pytest/issue/252/

Now on github: pytest-dev/pytest#252