some thoughts on meta fixtures and lazy fixtures

meta_fixture.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pytest and the Meta Fixture\n",
    "\n",
    "Meta fixtures, or fixtures that control the use/parametrization of other fixtures, are mentioned in several places (eg; [this issue](https://github.com/pytest-dev/pytest/issues/349), [this proposal](https://docs.pytest.org/en/latest/proposals/parametrize_with_fixtures.html), [this SO question of mine](https://stackoverflow.com/questions/34698922/output-of-fixture-in-params-in-pytest)), but pytest (as far as I can tell) currently doesn't have a clean, obvious way of doing this that works for all cases.\n",
    "\n",
    "Let me explain my use-case: I have several fixtures that all return similar yet different objects. There are some tests that should be run only on a specific fixture's output, and others that should be run on the outputs of all the fixtures. A meta-fixture to parametrize the outputs of the other fixtures seems like an elegant solution. Here is a completely contrived example: Let's assume we need to write some unit tests for python strings. We might have the following fixtures:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pytest\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def str_from_int():\n",
    "    \"\"\" return a string from an int \"\"\"\n",
    "    return str(42)\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def str_from_letters():\n",
    "    \"\"\" return a str from letters only \"\"\"\n",
    "    return 'abc'\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def empty_str():\n",
    "    \"\"\" return an empty str \"\"\"\n",
    "    return ''\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def none_str():\n",
    "    \"\"\" return str rep of None \"\"\"\n",
    "    return str(None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we might have some tests that only should run on strings from ints, some that should only run on strings from letters, and so on. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TestStrInts:\n",
    "    \"\"\" tests for str returned from ints \"\"\"\n",
    "    def test_is_numeric(self, str_from_int):\n",
    "        \"\"\" ensure a str rep of an in is numeric \"\"\"\n",
    "        assert str_from_int.isnumeric()\n",
    "\n",
    "\n",
    "class TestStrLetters:\n",
    "    \"\"\" tests for strs returned from pure letters \"\"\"\n",
    "    def test_is_not_numeric(self, str_from_letters):\n",
    "        \"\"\" should not be numeric \"\"\"\n",
    "        assert not str_from_letters.isnumeric()\n",
    "        \n",
    "    def test_is_alphanumeric(self, str_from_letters):\n",
    "        \"\"\" should be alpha numeric \"\"\"\n",
    "        assert str_from_letters.isalnum()\n",
    "\n",
    "\n",
    "class TestEmptyStr:\n",
    "    \"\"\" tests for empty strs \"\"\"\n",
    "    def test_bool_is_false(self, empty_str):\n",
    "        \"\"\" tests that bool rep of empty str is false \"\"\"\n",
    "        assert not empty_str"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But there are also some tests that should run on all string fixtures. The cleanest approach for this would be to combine all these fixtures into a single parametrized fixture, let's call it \"general_str\". "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class TestStrGeneral:\n",
    "    \"\"\" general tests that should pass for ALL str fixtures \"\"\"\n",
    "    expected_methods = ('strip', 'center', 'format', 'index', 'replace',\n",
    "                        'title')  # etc.\n",
    "\n",
    "    def test_has_expected_methods(self, general_str):\n",
    "        \"\"\" ensure strs have expected methods \"\"\"\n",
    "        for method_name in self.expected_methods:\n",
    "            assert hasattr(general_str, method_name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Simply defining the \"general_str\" fixture in this way, however, wont work:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@pytest.fixture(params=(str_from_int, str_from_letters, empty_str, none_str))\n",
    "def general_str(request):\n",
    "    \"\"\" meta fixture for collect all fixtures that return strs \"\"\"\n",
    "    return request.param"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "because request.param returns a reference to the fixture function itself, not its output. A fairly straight-forward work-around, based on [this comment](https://github.com/pytest-dev/pytest/issues/349#issuecomment-189370273) is pass a sequence of fixture names as strings to params, then to use the getfuncargvalue, or the more modern getfixturevalue, attribute of the request object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "str_fixtures = ('str_from_int', 'str_from_letters', 'empty_str', 'none_str')\n",
    "\n",
    "@pytest.fixture(params=str_fixtures)\n",
    "def general_str(request):\n",
    "    \"\"\" meta fixture for collect all fixtures that return strs \"\"\"\n",
    "    return request.getfixturevalue(request.param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now our tests will run as intended. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, there are some limitations. Mainly, if one of the base fixtures is itself a parametrized fixture:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "@pytest.fixture(params=range(100))\n",
    "def str_from_int(request):\n",
    "    \"\"\" return a string from an int \"\"\"\n",
    "    return str(request.param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will get an error with the message: :Failed: The requested fixture has no parameter defined for the current test.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At first glance I thought [pytest-lazy-fixture](https://github.com/TvoroG/pytest-lazy-fixture) would solve this problem, so I installed it and rewrote the fixtures like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# --- helper functions\n",
    "\n",
    "\n",
    "def lazify(*args):\n",
    "    \"\"\" return a list of lazy fixtures from args \"\"\"\n",
    "    return [pytest.lazy_fixture(x) for x in args]\n",
    "\n",
    "\n",
    "# --- Module level fixtures\n",
    "\n",
    "\n",
    "@pytest.fixture(params=range(100))\n",
    "def str_from_int(request):\n",
    "    \"\"\" return a string from an int \"\"\"\n",
    "    return str(request.param)\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def str_from_letters():\n",
    "    \"\"\" return a str from letters only \"\"\"\n",
    "    return 'abc'\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def empty_str():\n",
    "    \"\"\" return an empty str \"\"\"\n",
    "    return ''\n",
    "\n",
    "\n",
    "@pytest.fixture\n",
    "def none_str():\n",
    "    \"\"\" return str rep of None \"\"\"\n",
    "    return str(None)\n",
    "\n",
    "\n",
    "str_fixtures = ('str_from_int', 'str_from_letters', 'empty_str', 'none_str')\n",
    "\n",
    "@pytest.fixture(params=lazify(str_fixtures))\n",
    "def general_str(request):\n",
    "    \"\"\" meta fixture for collect all fixtures that return strs \"\"\"\n",
    "    return request.getfixturevalue(request.param)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, I get the same error message, so maybe it doesn't solve this issue? I will dig into it more when I get a chance. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}