|An alternative Django ``TEST_RUNNER`` which uses unittest2 test discovery from|
|a base path specified in settings, rather than requiring all tests to be in|
|``tests`` module of an app.|
|If you just run ``./manage.py test``, it'll discover and run all tests|
|underneath the ``TEST_DISCOVERY_ROOT`` setting (a path). If you run|
|``./manage.py test full.dotted.path.to.test_module``, it'll run the tests in|
|that module (you can also pass multiple modules).|
|And (new in this updated version), if you give it a single dotted path to a|
|package, and that package does not itself directly contain any tests, it'll do|
|test discovery in all sub-modules of that package.|
|This code doesn't modify the default unittest2 test discovery behavior, which|
|only searches for tests in files named "test*.py".|
|from django.conf import settings|
|from django.test import TestCase|
|from django.test.simple import DjangoTestSuiteRunner, reorder_suite|
|from django.utils.importlib import import_module|
|from django.utils.unittest.loader import defaultTestLoader|
|"""A test suite runner that uses unittest2 test discovery."""|
|def build_suite(self, test_labels, extra_tests=None, **kwargs):|
|suite = None|
|discovery_root = settings.TEST_DISCOVERY_ROOT|
|suite = defaultTestLoader.loadTestsFromNames(test_labels)|
|# if single named module has no tests, do discovery within it|
|if not suite.countTestCases() and len(test_labels) == 1:|
|suite = None|
|discovery_root = import_module(test_labels).__path__|
|if suite is None:|
|suite = defaultTestLoader.discover(|
|for test in extra_tests:|
|return reorder_suite(suite, (TestCase,))|
|You need the ``BASE_PATH`` and ``TEST_DISCOVERY_ROOT`` settings in order for|
|this test runner to work.|
|``BASE_PATH`` should be the directory containing your top-level package(s); in|
|other words, the directory that should be on ``sys.path`` for your code to|
|import. This is the directory containing ``manage.py`` in the new Django 1.4|
|``TEST_DISCOVERY_ROOT`` should be the root directory to discover tests|
|within. You could make this the same as ``BASE_PATH`` if you want tests to be|
|discovered anywhere in your project. If you want tests to only be discovered|
|within, say, a top-level ``tests`` directory, you'd set ``TEST_DISCOVERY_ROOT``|
|as shown below.|
|And you need to point the ``TEST_RUNNER`` setting to the ``DiscoveryRunner``|
|# This is correct for the Django 1.4-style project layout; for the old-style|
|# project layout with ``settings.py`` and ``manage.py`` in the same directory,|
|# you'd want to only call ``os.path.dirname`` once.|
|BASE_PATH = os.path.dirname(os.path.dirname(__file__))|
|# This would be if you put all your tests within a top-level "tests" package.|
|TEST_DISCOVERY_ROOT = os.path.join(BASE_PATH, "tests")|
|# This assumes you place the above ``DiscoveryRunner`` in ``tests/runner.py``.|
|TEST_RUNNER = "tests.runner.DiscoveryRunner"|
@pydanny - you talking about this? https://github.com/opencomparison/opencomparison/blob/master/testrunner.py
It looks like that just defaults to a settings-defined set of project apps instead of all installed apps, yeah?
One issue with this that I ran into is when specifying app_labels. The problem is that
I fixed it by just calling the parent
@streeter - Indeed. That's not a bug, that's a feature :-) Selecting tests by full dotted path rather than by app label is part of the purpose of this alternative runner. It is more flexible and gives more control. For one thing, it allows keeping tests in any Python package/module, not just in an installed app. And if tests are split into multiple modules within a tests/ package in an app, it allows running just the tests in one module, rather than forcing you to either run all the tests in an app, or just a single TestCase. And, of course, it's the way every other Python test runner works :-)
In a broader sense, I consider app-labels to be a mistake in Django's design in the first place. They take a functional and well-known namespace, the Python module namespace, and attempt to "flatten" it. This just causes needless collisions (inability to have both django.contrib.auth and my.own.auth) and doesn't really offer any benefits.
There is discussion ongoing to perhaps make something along these lines the default test runner in Django 1.4 (with a few more options added to allow selecting tests by app label for those who prefer that). You may want to take a look and weigh in if you're interested: https://code.djangoproject.com/ticket/17365
Ah, interesting. I hadn't thought that was a feature :) However, it definitely makes it more flexible at the expense of a bit of extra typing. Though when I'm typing individual app labels, I'm also using my bash history more often. I'll check out the 1.4 ticket and think about it a bit more.
@carljm - I'm trying to split out tests into separate files as per your PyCon 2012 talk, but I'm not able to get this working in Django 1.3.1. All of the following command find zero tests:
My tests are in
@chase-seibert Probably your layout is the old-style project layout (with manage.py and settings.py adjacent), in which case there may be one too many
The other issue is a known limitation of the simplified runner code; it either does full discovery, or runs exactly the modules you give it; it doesn't do partial discovery within a given module/package. The updated version of this gist I just posted (which was at the /code link in my talk slides, but didn't fit on a slide itself :-) adds in the partial-discovery feature.
Thanks for the help, Carl. I've tried creating a new project under Django 1.4c1. I've uploaded it here: http://dl.dropbox.com/u/422013/testsite.tar.gz
(virtualenv)chase@chase-mint ~/testsite $ ./manage.py test BASE_PATH: /home/chase/testsite TEST_DISCOVERY_ROOT: /home/chase/testsite/tests Creating test database for alias 'default'... ---------------------------------------------------------------------- Ran 0 tests in 0.000s OK Destroying test database for alias 'default'... (virtualenv)chase@chase-mint ~/testsite $ ./manage.py test tests BASE_PATH: /home/chase/testsite TEST_DISCOVERY_ROOT: /home/chase/testsite/tests Creating test database for alias 'default'... TestCase1 . ---------------------------------------------------------------------- Ran 1 test in 0.000s OK Destroying test database for alias 'default'... (virtualenv)chase@chase-mint ~/testsite $ ./manage.py test tests.more BASE_PATH: /home/chase/testsite TEST_DISCOVERY_ROOT: /home/chase/testsite/tests Creating test database for alias 'default'... TestCase2 . ---------------------------------------------------------------------- Ran 1 test in 0.001s OK Destroying test database for alias 'default'...
I'm expecting the first and second commands to find both tests, the one in
@chase-seibert Ah, I didn't think to document how unittest2 test discovery itself works. By default, only files named
Firstly, thanks for this. When I'm more hip I might use nose or something but this works perfectly.
Secondly there is a tiny mismatch between runner.py and settings.py.
Thanks @mallison, fixed.
Nose (with django-nose, so you don't lose the Django db setup/teardown stuff) is a great option, too. The main reason for this gist is that it's pre-testing something we're hoping to put in Django core in time for 1.5, and requiring nose isn't really an option for core.
In case anyone is interested, I've packaged this up for general consumption (apologies @carljm, I needed it quick): http://pypi.python.org/pypi/django-discover-runner
I'm stuck on how to use this with Jenkins. If I want my Jenkins server to use this plugin, then I replace the "./manage.py jenkins" command that Jenkins was running with "./manage.py test". But now I'm no longer using the django-jenkins plugin command, I don't get the XML output that I think Jenkins needs in order to display the test results. Am I being dense?