Skip to content

Instantly share code, notes, and snippets.

@asears
Last active September 21, 2022 11:35
Show Gist options
  • Save asears/01a299c26433f50fcaf36cfc68ceb313 to your computer and use it in GitHub Desktop.
Save asears/01a299c26433f50fcaf36cfc68ceb313 to your computer and use it in GitHub Desktop.
Unit testing with Databricks

Unit Testing with Databricks

pytest-mock

sys.dont_write_bytecode = True # or set init script to allow writing to repo for pytest.
dbutils.library.restartPython() # to clear the interpreter before a test.

Find the repo root

notebook_path = dbutils.notebook.entry_point.getDbutils().notebook().getContext().notebookPath().get()
repo_root = os.path.dirname(os.path.dirname(notebook_path))
os.chdir(f'/Workspace/{repo_root}')
retcode = pytest.main([".", "-p", "no:cacheprovider"])

# Fail the cell execution if we have any test failures.
assert retcode == 0, 'The pytest invocation failed. See the log above for details.'

https://github.com/databricks/notebook-best-practices/blob/main/notebooks/run_unit_tests.py

Writing test outputs to the FileStore

import pytest

retcode = pytest.main(['-k', 'mycode_test',
                       '-o', 'cache_dir=/dbfs/FileStore/',
                       '--junitxml', '/dbfs/FileStore/pytestreport.xml',
                       '-v'
                      ]
                     )
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment