Skip to content

Instantly share code, notes, and snippets.

@tdhopper
Created June 29, 2020 18:12
Show Gist options
  • Save tdhopper/9fff3236d8504bd0ab1358981a92e744 to your computer and use it in GitHub Desktop.
Save tdhopper/9fff3236d8504bd0ab1358981a92e744 to your computer and use it in GitHub Desktop.
data/
.env/
__pycache__
.ipynb_checkpoints
Training Report.pdf
*.tar.gz
handler_venv/
*.nbconvert.*
static/
env/
*.pkl
.env
.ipynb_checkpoints
**/__pycache__

Energy Efficiency Predictor

Problem

The goal of this challenge is to build a regression model and deploy it with docker. The dataset you will use for the challenge is available at https://archive.ics.uci.edu/ml/datasets/Energy+efficiency. You should be able to run the docker image and then curl the container by sending json containing the attributes of a new building and get a json response with the heating and cooling loads predicted by your trained model. The code should be written in python but you can use whichever libraries you like to train and deploy the model

Solution

I built a simple multivariate, Lasso model with Scikit-learn that is served with Flask.

You can see the notebook used to train the model at https://9whioydhmb.execute-api.us-east-1.amazonaws.com/carbonrelay.

Model can be tested by posting a dictionary of input values to an AWS endpoint, e.g.:

curl -d '{
    "relative_compactness": 0.98,
    "surface_area": 514.5,
    "wall_area": 294.0,
    "roof_area": 110.25,
    "overall_height": 7.0,
    "orientation": 2.0,
    "glazing_area": 0.0,
    "glazing_area_distribution": 0.0
}' -H 'Content-Type: application/json' https://9whioydhmb.execute-api.us-east-1.amazonaws.com/carbonrelay/predict

HTTP response is JSON containing an heating_load and cooling_load field.

Building and Testing

Prerequisites

  • Running Docker client

Steps

Run $ docker-compose up to train model and open webserver on port 5000. Once this completes, you should be able to run:

curl -d '{
    "relative_compactness": 0.98,
    "surface_area": 514.5,
    "wall_area": 294.0,
    "roof_area": 110.25,
    "overall_height": 7.0,
    "orientation": 2.0,
    "glazing_area": 0.0,
    "glazing_area_distribution": 0.0
}' -H 'Content-Type: application/json' http://127.0.0.1:5000/predict

You can also view the Jupyter notebook with training information and model performance information at http://127.0.0.1:5000/.

import joblib
import pandas as pd
from flask import Flask, request
app = Flask("energy_efficiency")
@app.route("/")
def report():
return app.send_static_file("Train.html")
@app.route("/predict", methods=["POST"])
def predict():
data = pd.DataFrame.from_records([request.get_json()])
pipe = joblib.load("model.pkl")
output = pipe.predict(data)
assert output.shape == (1, 2)
return {
"heating_load": output[0][0],
"cooling_load": output[0][1],
}
version: '3'
services:
web-server:
build: .
entrypoint: flask run -h 0.0.0.0
ports:
- "5000:5000"
volumes:
- .:/home
train:
build: .
entrypoint: /bin/sh
command: /home/train.sh
volumes:
- .:/home
FROM fnndsc/ubuntu-python3
RUN apt -qq install --yes build-essential
RUN pip install --quiet --upgrade pip
WORKDIR /home
COPY . ./
RUN pip install -r requirements.txt
ENV FLASK_APP=app
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
scikit-learn
flask
jupyter
pandas
xlrd
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
#!/bin/bash
jupyter nbconvert --to html --execute Train.ipynb --ExecutePreprocessor.timeout=600
mkdir -p static
mv Train.html static
{
"carbonrelay": {
"app_function": "app.app",
"aws_region": "us-east-1",
"project_name": "energy_efficiency",
"runtime": "python3.8",
"s3_bucket": "zappa-carbon-relay-takehome",
"slim_handler": true
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment