Amazon Sagemaker + Nerdwallet ML Platform - Blog Post

local-development.txt

> nwml train
Starting local training for nwpy-ml-iris
Loading the ml.train entry point...
  * Success!
...
Done!

> nwml batch-predict
...
Done!

ml.yml

...

resources:
    instance_type: ml.m5.xlarge
    instance_count: 1
    volume_size: 1

input_data_files:
  - name: train
    source: s3_prefix
    uri: <transit-bucket>/test-data/iris/train
    distribution: full

new-ml-pipeline-structure.txt

> tree
.
├── Jenkinsfile
├── README.md
├── VERSION
├── app.yml
├── data
│   ├── batch-predict
│   │   └── raw
│   │       └── sample.txt
│   └── train
│       └── raw
├── ml.yml
├── requirements.txt
├── setup.cfg
├── setup.py
└── src
    └── nwml
        ├── __init__.py
        └── iris
            ├── __init__.py
            └── model.py

new-ml-pipeline.txt

> indy new python-ml-lib
name [your-lib]: iris
module_namespace [yourlib]: iris
description [Test ML Pipeline]:
...
Done!

setup.py

from setuptools import setup

setup(
 # ...
 entry_points={
  # ...
  "ml": [
    "batch-predict = nwml.iris.model:batch_predict",
    "load-model = nwml.iris.model:load_model",
    "save-model = iris.loft.model:save_model",
    "train = nwml.iris.model:train",
  ],

 }
)

src.nwml.iris.model.py

def train(params, inputs, output_dir):
    """Train a model from inputs and hyper parameters"""
    pass


def save_model(model_data, output_dir):
    """Serialize the model returned by `train` to disk"""
    pass


def load_model(output_dir):
    """Load a model from the folder written by `save_model`"""
    pass


def batch_predict(model_data, inputs, output_dir):
    """
    Make batch predictions from a model (from `train` or `load_model`),
    and some inputs, and write to disk.
    """
    pass