I was wrong and Alex was right.

README.md

I was wrong and Alex was right.

I'm settling a friendly debate with my friend, Alex. I was wrong and he was right.

You're telling me you can execute 20 parametrized test HTTP calls to a Flask service with full database setup/teardown in under a second?

Yes.

Show me.

I doubted, but no more. It is definitely possible to execute those Flask tests in under a second (as long as you don't count container startup time).

This is obviously a toy application and depending on the scope of the pytest fixture it's not always under a second, but it's usually within an acceptable threshold. If you set the fixture scope to module it's basically always under a second, though that does risk considerable test impurity.

Instructions

First, start the DB and build the test container.

docker-compose up -d db && docker-compose build test

Then, run the tests with function scope (the default).

time SCOPE=function docker-compose up --no-log-prefix test
Recreating fast-tests_test_1 ... done
Attaching to fast-tests_test_1
============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /test
collected 20 items

app.py ....................                                              [100%]

============================== 20 passed in 0.90s ==============================
fast-tests_test_1 exited with code 0
SCOPE=function docker-compose up --no-log-prefix test  0.40s user 0.05s system 19% cpu 2.299 total

Finally, run the tests again with module scope just to see the improvement.

time SCOPE=module docker-compose up --no-log-prefix test
Recreating fast-tests_test_1 ... done
Attaching to fast-tests_test_1
============================= test session starts ==============================
platform linux -- Python 3.9.7, pytest-6.2.5, py-1.10.0, pluggy-1.0.0
rootdir: /test
collected 20 items

app.py ....................                                              [100%]

============================== 20 passed in 0.46s ==============================
fast-tests_test_1 exited with code 0
SCOPE=module docker-compose up --no-log-prefix test  0.42s user 0.05s system 24% cpu 1.974 total

app.py

import os

import psycopg2
import pytest
from flask import Flask, current_app, g

SCOPE = os.getenv("SCOPE", "function")
DB_URI = os.getenv("DB_URI", "postgresql://postgres:postgres@localhost/postgres")

DELETE = "DROP TABLE IF EXISTS names;"
CREATE = "CREATE TABLE IF NOT EXISTS names (id SERIAL PRIMARY KEY, name TEXT);"


def get_db():
    if "db" not in g:
        g.db = psycopg2.connect(current_app.config["DB_URI"])
    return g.db


def close_db(e=None):
    db = g.pop("db", None)
    if db is not None:
        db.close()


def init_db():
    db = get_db()
    with db.cursor() as cursor:
        cursor.execute(DELETE)
        cursor.execute(CREATE)
        db.commit()


def create_app(db_uri=DB_URI):
    app = Flask(__name__)
    app.config.from_mapping(DB_URI=db_uri)
    app.teardown_appcontext(close_db)

    @app.get("/hello/<name>")
    def hello(name):
        db = get_db()
        with db.cursor() as cursor:
            cursor.execute(
                "INSERT INTO names (name) VALUES (%s) RETURNING name", (name,)
            )
            result = cursor.fetchone()[0]
        return f"Hello, {result}!"

    return app


@pytest.fixture(scope=SCOPE)
def client():
    app = create_app()
    with app.test_client() as client:
        with app.app_context():
            init_db()
        # This isn't a perfect fixture. It doesn't clean the DB on teardown.
        yield client


@pytest.mark.parametrize(
    "name",
    (
        "Liam",
        "Olivia",
        "William",
        "Emma",
        "Ramón",
        "María",
        "محمد",
        "مريم",
        "莉子",
        "陽葵",
        "민준",
        "서연",
        "家豪",
        "淑芬",
        "Դավիթ",
        "Նարե",
        "Ανδρέας",
        "Μαρία",
        "Александр",
        "София",
    ),
)
def test_hello(client, name):
    response = client.get(f"/hello/{name}").data.decode("utf-8")
    assert response == f"Hello, {name}!"

docker-compose.yml

version: "3.9"
services:
  db:
    image: postgres:latest
    restart: always
    ports:
      - "5432:5432"
    environment:
      POSTGRES_PASSWORD: postgres
  test:
    build: .
    environment:
      - DB_URI=postgresql://postgres:postgres@db/postgres
      - SCOPE=${SCOPE:-function}

Dockerfile

FROM python:latest

WORKDIR /test

ADD ./app.py /test

RUN pip install flask psycopg2-binary pytest

CMD ["pytest", "app.py"]