Skip to content

Instantly share code, notes, and snippets.

@HemachandranD
Last active March 8, 2024 17:27
Show Gist options
  • Save HemachandranD/177b549119eb16544978f40df6c9443a to your computer and use it in GitHub Desktop.
Save HemachandranD/177b549119eb16544978f40df6c9443a to your computer and use it in GitHub Desktop.
my_mlops_stacks <- Root directory. Both monorepo and polyrepo are supported.
├── my_mlops_stacks <- Contains python code, notebooks and ML resources related to one ML project.
│ │
│ ├── requirements.txt <- Specifies Python dependencies for ML code (for example: model training, batch inference).
│ │
│ ├── databricks.yml <- databricks.yml is the root bundle file for the ML project that can be loaded by databricks CLI bundles. It defines the bundle name, workspace URL and resource config component to be included.
│ │
│ ├── training <- Training folder contains Notebook that trains and registers the model with feature store support.
│ │
│ ├── feature_engineering <- Feature computation code (Python modules) that implements the feature transforms.
│ │ The output of these transforms get persisted as Feature Store tables. Most development
│ │ work happens here.
│ │
│ ├── validation <- Optional model validation step before deploying a model.
│ │
│ │
│ ├── deployment <- Deployment and Batch inference workflows
│ │ │
│ │ ├── batch_inference <- Batch inference code that will run as part of scheduled workflow.
│ │ │
│ │ ├── model_deployment <- As part of CD workflow, deploy the registered model by assigning it the appropriate alias.
│ │
│ │
│ ├── tests <- Unit tests for the ML project, including the modules under `features`.
│ │
│ ├── resources <- ML resource (ML jobs, MLflow models) config definitions expressed as code, across dev/staging/prod/test.
│ │
│ ├── model-workflow-resource.yml <- ML resource config definition for model training, validation, deployment workflow
│ │
│ ├── batch-inference-workflow-resource.yml <- ML resource config definition for batch inference workflow
│ │
│ ├── feature-engineering-workflow-resource.yml <- ML resource config definition for feature engineering workflow
│ │
│ ├── ml-artifacts-resource.yml <- ML resource config definition for model and experiment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment