Skip to content

Instantly share code, notes, and snippets.

@ClementWalter
ClementWalter / tf_serving_heroku.md
Created Sep 23, 2020
How to deploy a tensorflow model on Heroku with tensorflow serving
View tf_serving_heroku.md

How to deploy a tensorflow model on Heroku with tensorflow serving

After spending minutes or hours playing around with all the wonderful examples available for instance on the Google AI hub, one may wants to deploy one model or another online.

This article presents a fast, optimal and neat way of doing it with Tensorflow Serving and Heroku.

Introduction

@ClementWalter
ClementWalter / tf_serving_entrypoint.sh
Created Sep 23, 2020
Modified to take $PORT env variable
View tf_serving_entrypoint.sh
#!/bin/bash
tensorflow_model_server --port=8500 --rest_api_port="${PORT}" --model_name="${MODEL_NAME}" --model_base_path="${MODEL_BASE_PATH}"/"${MODEL_NAME}" "$@"
@ClementWalter
ClementWalter / Dockerfile
Last active Sep 23, 2020
Tf servint Heroku dockerfile
View Dockerfile
FROM tensorflow/serving
ENV MODEL_BASE_PATH /models
ENV MODEL_NAME classifier
COPY models/classifier /models/classifier
# Fix because base tf_serving_entrypoint.sh does not take $PORT env variable while $PORT is set by Heroku
# CMD is required to run on Heroku
COPY tf_serving_entrypoint.sh /usr/bin/tf_serving_entrypoint.sh
@ClementWalter
ClementWalter / batch_request_served_model.py
Created Sep 23, 2020
Perform a batch request onto a Tensorflow served model with docker
View batch_request_served_model.py
response = requests.post(
"http://localhost:8501/v1/models/classifier:predict",
json={
"signature_name": "serving_default", # can be omitted
"inputs": {
"image_bytes": [image.numpy().decode("utf-8") for image in image_bytes][:2], # batch request
},
},
)
View basic_saved_model.py
tf.saved_model.save(
classifier,
export_dir="classifier/1",
signatures={
"serving_default": decode_and_serve,
"preprocessing": preprocessing,
},
)
@ClementWalter
ClementWalter / decode_and_serve.py
Created Sep 23, 2020
Signatures to decode a base64 image and serve inference
View decode_and_serve.py
@tf.function(input_signature=(tf.TensorSpec(shape=[None], dtype=tf.string),))
def decode(image_bytes):
"""
Takes a base64 encoded image and returns the preprocessed input tensor ready for the inference
"""
# not working on GPU if tf.__version__ < 2.3, see https://github.com/tensorflow/tensorflow/issues/28007
with tf.device("/cpu:0"):
input_tensor = tf.map_fn(
lambda x: preprocessing(tf.io.decode_jpeg(contents=tf.io.decode_base64(x), channels=3))["output_0"],
image_bytes,
@ClementWalter
ClementWalter / preprocessing.py
Created Sep 23, 2020
Simple image preprocessing
View preprocessing.py
@tf.function(input_signature=(tf.TensorSpec(shape=[None, None, 3], dtype=tf.uint8),))
def preprocessing(input_tensor):
output_tensor = tf.cast(input_tensor, dtype=tf.float32)
output_tensor = tf.image.resize_with_pad(output_tensor, target_height=224, target_width=224)
output_tensor = keras_applications.mobilenet.preprocess_input(output_tensor, data_format="channels_last")
return output_tensor
View machine_learning_principles.md

5 things you need to know before starting your machine learning project

and why Jupyter Notebook is not a good tool to do it.

At Sicara, we build machine learning based products for our customers:

  • we build products: we need to develop in a production-ready mindset. Algorithms are served in the cloud, served and updated with APIs, etc.
  • machine learning products: the customer comes with a business need and we have to deliver a satisfying solution as fast as possible.
@ClementWalter
ClementWalter / scratch.md
Created Feb 14, 2019
Perfect notebook with PyCharm and Pweave
View scratch.md

The perfect notebook is not a jupyter one

Jupyter notebook has been heavily reported as the perfect prototyping tool for data scientist. Its main features are:

  • inline code execution
  • easy idea structuring
  • nice displays of pictures and dataframe

This overall flexibility has made it a preferred tool compared to the more rustic iPython command line. However it should not be forgotten that this is no more than an REPL where you can navigate efficiently throughout the history.

You can’t perform that action at this time.