https://www.python.org/dev/peps/pep-0008/, and Google Style Guide but generally speaking -- be consistent, comment often, and writer readable code:
- Favor additional whitespace for readability: (x + y) vs. (x+y)
- Trailing commas are OK: [a, b, c, d,]
- short, lowercase names for packages/modules
- CapWords for classes
- lower_case for functions and variables
- https://www.python.org/dev/peps/pep-0257/
- Numpy-style docstrings (good example)
- http://www.writethedocs.org/guide/writing/beginners-guide-to-docs/
- https://technology.cloverhealth.com/a-data-scientists-guide-to-writing-unit-tests-ff1dd716aabc
- https://pytest.org
- https://github.com/htorrence/pytest_examples
- https://eugeneyan.com/writing/setting-up-python-project-for-automation-and-collaboration/
- https://eugeneyan.com/writing/testing-ml/
- https://medium.com/python-in-plain-english/unit-testing-in-python-structure-57acd51da923
- RF/Decision Tree Importance plot
- Shapley Values
- LIME
- XAI (eXplainable AI)
- Install jq Steps: Download 'jq' (and note where you saved it). For Windows, install with chocolatey. (*nix) Add the following to ~/.gitconfig
[core]
attributesfile = ~/.gitattributes_global
[filter "nbstrip_full"]
clean = "jq --indent 1 \
'(.cells[] | select(has(\"outputs\")) | .outputs) = [] \
| (.cells[] | select(has(\"execution_count\")) | .execution_count) = null \
| .metadata = {\"language_info\": {\"name\": \"python\", \"pygments_lexer\": \"ipython3\"}} \
| .cells[].metadata = {} \
'"
smudge = cat
required = true
(windows) Add the following to C:\Users<username>.gitconfig . Alter the path at "clean = " for your 'jq' save location.
[core]
attributesfile = C:\\Users\\<username>\\.gitattributes_global
[filter "nbstrip_full"]
clean = "jq --indent 1 \
'(.cells[] | select(has(\"outputs\")) | .outputs) = [] \
| (.cells[] | select(has(\"execution_count\")) | .execution_count) = null \
| .metadata = {\"language_info\": {\"name\": \"python\", \"pygments_lexer\": \"ipython3\"}} \
| .cells[].metadata = {} \
'"
smudge = cat
required = true
Add the following to ~/.gitattributes_global or C:\Users<username>.gitattributes_global
*.ipynb filter=nbstrip_full
References:
- http://timstaley.co.uk/posts/making-git-and-jupyter-notebooks-play-nice/
- https://nms.kcl.ac.uk/john.armstrong/misc/jupyter-and-git.html
- https://www.theserverside.com/blog/Coffee-Talk-Java-News-Stories-and-Opinions/Where-system-global-and-local-Windows-Git-config-files-are-saved
- CausalNex - Causal Modeling
[MLFlow] Kedro Kedro + MLFlow Workflow discussion/example Aggregator References: Awesome Production ML Production Level Deep Learning Lessons Learned