Python's unittest package rundown

python_unittest.md

Unittest

Concepts:

• Test Fixtures: setup and teardown
• Test Case: unit of testing
• Test Suite: group of test and/or suites
• Test Runner: orchestrates execution of tests

methods, functions name must start with test (default)

class name need not start with test

test are sorted using their names (default)

setup(), teardown() and init() are called for each testcase

Package: import unittest

• Class based test: class TestClass(unittest.TestCase)
• Function based test: def test_fuction()
• Running test: unittest.main()
• Converting old functional testcases: unittest.FunctionalTestCase(testMethod, setUp=initialize, teardown=cleanUp)

CLI:

python -m unittest test_module1 test_module2
python -m unittest test_module.TestClass
python -m unittest test_module.TestClass.test_method

CLI Options:

• -b, --buffer
• -c, --catch: Control-C during test run waits for current test to end
• -f, --failfast: stop test run on first error or failure
• -k: only run specified test patterns
• --locals: show local variables in tracebacks

Test Discovery

#both are same
python -m unittest discover test_module.TestClass
python -m unittest test_module.TestClass

Options for discover:

• -v, --verbose: verbose output
• -s, --start-directory: specify start directory (. default)
• -p, --pattern: pattern to match test files
• -t, --top-level-directory: top level directory

Skipping test & expected failures

Decorators:

• @unittest.skip(msg)
• @unittest.skipIf(condition, msg)
• @unittest.skipUnless(condition, msg)
• @unittest.expectedfailure

Methods:

• skipTest(msg)

Subtest

with self.subTest(msg, **params):
    #assertions

Class unittest.TestCase:

Class methods:

• setUpClass()
• tearDownClass()
• addClassCleanup(function, /, *args, **kwargs)
• doClassCleanups()

Instance methods:

• setUp()
• tearDown()
• run(result=None): pass TestResult object
• skipTest(reason)
• subTest(msg=None, **params)
• debug(): run test without collecting result, debugging tests
• fail(msg=None)
• countTestCases()
• defaultTestResult()
• id()
• shortDescription()
• addCleanup(function, *args, **kwargs)
• doCleanups()

Attributes:

• failureException
• longMessage
• maxDiff

Assert methods:

All assert methods support msg as last argument

• assertEqual(a,b) | assertNotEqual(a,b)
• assertAlmostEqual(first, second, places=7, msg=None, delta=None) | assertNotAlmostEqual(first, second, places=7, msg=None, delta=None)
• assertGreater(first, second, msg=None) | assertGreaterEqual(first, second, msg=None)
• assertLess(first, second, msg=None) | assertLessEqual(first, second, msg=None)
• assertTrue(expr) | assertFalse(expr)
• assertIs(a,b) | assertIsNot(a,b)
• assertIsNone(expr) | assertIsNotNone(expr)
• assertCountEqual(first, second, msg=None)
• assertIn(member, container) | assertNotIn(member, container)
• assertRegex(text, regex, msg=None) | assertNotRegex(text, regex, msg=None)
• assertIsInstance(obj, cls) | assertNotIsInstance(obj, cls)
• assertMultiLineEqual(first, second, msg=None)
• assertSequenceEqual(first, second, msg=None, seq_type=None)
• assertListEqual(first, second, msg=None)
• assertTupleEqual(first, second, msg=None)
• assertSetEqual(first, second, msg=None)
• assertDictEqual(first, second, msg=None)
• assertRaises(exception, callable, *args, **kwds) | assertRaises(exception, *, msg=None)

with self.assertRaises(SomeException): 
    do_something()

• assertRaisesRegex(exception, regex, callable, *args, **kwds) | assertRaisesRegex(exception, regex, *, msg=None)

with self.assertRaisesRegex(ValueError, 'literal'): 
   int('XYZ')

• assertWarns(warning, callable, *args, **kwds) | assertWarns(warning, *, msg=None)

with self.assertWarns(SomeWarning): 
    do_something()

• assertWarnsRegex(warning, regex, callable, *args, **kwds) | assertWarnsRegex(warning, regex, *, msg=None)

with self.assertWarnsRegex(RuntimeWarning, 'unsafe frobnicating'): 
    frobnicate('/etc/passwd')

• assertLogs(logger=None, level=None)

with self.assertLogs('foo', level='INFO') as cm: 
   logging.getLogger('foo').info('first message')

Grouping tests: unittest.TestSuite(tests=())

• addTest(test)
• addTests(tests)
• run(result)
• debug()
• countTestCases()
• __iter__()

Loading and running tests: unittest.TestLoader

Methods:

• loadTestsFromTestCase(testCaseClass)
• loadTestsFromModule(module, pattern=None)
• loadTestsFromName(name, module=None)
• loadTestsFromNames(names, module=None)
• getTestCaseNames(testCaseClass)
• discover(start_dir, pattern='test*.py', top_level_dir=None)

Attributes:

• testMethodPrefix
• sortTestMethodsUsing
• suiteClass
• testNamePatterns

Result of tests: unittest.TestResult

Methods:

• wasSuccessful()
• stop()
• startTest(test)
• stopTest(test)
• startTestRun()
• stopTestRun()
• addError(test, err)
• addFailure(test, err)
• addSuccess(test)
• addSkip(test, reason)
• addExpectedFailure(test, err)
• addUnexpectedSuccess(test)
• addSubTest(test, subtest, outcome)

Attributes:

• errors
• failures
• skipped
• expectedFailures
• unexpectedSuccesses
• shouldStop
• testsRun
• buffer
• failfast
• tb_locals

TestResult's concrete implementation: unittest.TextTestResult(stream, descriptions, verbosity)

unittest.TextTestRunner(stream=None, descriptions=True, verbosity=1, failfast=False, buffer=False, resultclass=None, warnings=None, *, tb_locals=False)

unittest.main(module='__main__', defaultTest=None, argv=None, testRunner=None, testLoader=unittest.defaultTestLoader, exit=True, verbosity=1, failfast=None, catchbreak=None, buffer=None, warnings=None)

load_tests(loader, standard_tests, pattern): add this function to customize test loading

def load_tests(loader, tests, pattern): 
    suite = TestSuite() 
    for test_class in test_cases: 
        tests = loader.loadTestsFromTestCase(test_class) 
        suite.addTests(tests) 
    return suite

def load_tests(loader, standard_tests, pattern): 
    # top level directory cached on loader instance 
    this_dir = os.path.dirname(__file__) 
    package_tests = loader.discover(start_dir=this_dir, pattern=pattern) 
    standard_tests.addTests(package_tests) 
    return standard_tests

Module fixtures:

• setUpModule()
• tearDownModule()
• unittest.addModuleCleanup(function, /, *args, **kwargs)
• unittest.doModuleCleanups()

Signal Handling

• unittest.installHandler()
• unittest.registerResult(result)
• unittest.removeResult(result)
• unittest.removeHandler(function=None)