pytest-grouping-tests.md

topics

• inbuilt markers
• custom markers
  • marking classes or modules
• node id
• based on name
• --ignore-flag
• norecursedirs
• naming conventions

inbuilt markers

• it's possible to use inbuilt markers as xfail, skip (skips a test unconditionally), skipif and parametrize

custom markers

import pytest

@pytest.mark.api
def test_http_request(): 
   pass

@pytest.mark.api
def test_another_http_request():
   pass

@pytest.mark.script
def test_something_else():
   pass
   
class TestClass:
  def test_script_1(self):
     pass
     
  def test_script_1(self):
     pass

If you need to restrict a test run to only run tests marked with script:

$ pytest -v -m script

Or to run all tests except the api ones:

$ pytest -m "not script" -v

marking classes or modules

import pytest

@pytest.mark.script
class TestClass:
  def test_script_1(self):
     pass
     
  def test_script_1(self):
     pass

node ID

Use the form module.py::class::method or module.py::function.

$ pytest -v test_app.py::TestClass::test_i_dont_know_what

# select the class
$ pytest -v test_server.py::TestClass

# multiple nodes
$ pytest -v test_server.py::TestClass test_server.py::test_another_http_request

based on name

$ pytest -v -k _script_ 

Run all tests expect the ones that match the keyword.

$ pytest -k "not _script_" -v

--ignore flag

We can use the flag --ignore=path to specify certain test directories and modules.

'-- scripts/
|   |-- tests
|   |-- test_script_01.py
|   '-- test_script_02.py
'-- api/
    '-- tests
    |   |-- test_api_01.py
    |   '-- test_api_02.py
    '-- loadtest
        |-- test_loadtest_01.py
        '-- test_loadtest_02.py

$ pytest --ignore=scripts/tests --ignore=api/tests/test_loadtest_01.py --ignore=api/loadtests/

We can add this command in pytest.ini.

[pytest]
addopts = --ignore=scripts/tests --ignore=api/tests/test_loadtest_01.py --ignore=api/loadtests/

norecursedirs

#TODO: test this Maybe another option is the norecursedirs. We can set the directory basename pattern to avoid.

   *       matches everything
   ?       matches any single character
   [seq]   matches any character in seq
   [!seq]  matches any char not in seq

naming conventions

We also can define naming conventions, using python_files, python_classes and python_functions.

refs

• https://www.tutorialspoint.com/pytest/pytest_grouping_the_tests.htm
• https://docs.pytest.org/en/6.2.x/example/markers.html
• https://realpython.com/pytest-python-testing/#marks-categorizing-tests
• https://stackoverflow.com/questions/36456920/is-there-a-way-to-specify-which-pytest-tests-to-run-from-a-file
• https://stackoverflow.com/questions/11117062/how-to-tell-py-test-to-skip-certain-directories
• https://docs.pytest.org/en/stable/example/pythoncollection.html#ignore-paths-during-test-collection