Skip to content

Instantly share code, notes, and snippets.

View afranzi's full-sized avatar
💭
🦊

Albert Franzi afranzi

💭
🦊
23 followers · 3 following
View GitHub Profile
@afranzi
afranzi / genesis_public_key
Created February 21, 2018 11:39
044865cb6f588c41fbcaad474ae636f2a57e8dcb4e514eb9ac93cc8f063016c167814032c2461b2050878e5d9b867722fe93ceeb85c873cec8d0f01fd40bde81b8;rogererill
@afranzi
afranzi / SetupMLflow.md
Last active October 23, 2018 21:24
Setting up MLflow 
pyenv install 3.7.0
pyenv global 3.7.0 # Use Python 3.7
mkvirtualenv mlflow # Create a Virtual Env with Python 3.7
workon mlflow

Install required libraries.

pip install mlflow==0.7.0 \
@afranzi
afranzi / runTrackingServer.sh
Created October 23, 2018 21:25
# Running a Tracking Server
mlflow server \
--file-store /tmp/mlflow/fileStore \
--default-artifact-root s3://<bucket>/mlflow/artifacts/ \
--host localhost
--port 5000
@afranzi
afranzi / runWineModel.sh
Created October 23, 2018 21:30
Launch wine quality prediction model with MLflow.
MLFLOW_TRACKING_URI=http://localhost:5000 python wine_quality.py \
--alpha 0.9
--l1_ration 0.5
--wine_file ./data/winequality-red.csv
@afranzi
afranzi / wineModelLogging.py
Created October 23, 2018 21:34
Log model by using the MLflow Tracking Server
with mlflow.start_run():
... model ...
mlflow.log_param("source", wine_path)
mlflow.log_param("alpha", alpha)
mlflow.log_param("l1_ratio", l1_ratio)
mlflow.log_metric("rmse", rmse)
mlflow.log_metric("r2", r2)
@afranzi
afranzi / wineQualityPredictionCurl.sh
Created October 23, 2018 21:46
# Query Tracking Server Endpoint
# Query Tracking Server Endpoint
curl -X POST \
http://127.0.0.1:5005/invocations \
-H 'Content-Type: application/json' \
-d '[
{
"fixed acidity": 3.42,
"volatile acidity": 1.66,
"citric acid": 0.48,
"residual sugar": 4.2,
@afranzi
afranzi / pyspark.sh
Created October 23, 2018 22:00
Launch PySpark with Jupyter
(mlflow) afranzi:~$ pyspark
[I 19:05:01.572 NotebookApp] sparkmagic extension enabled!
[I 19:05:01.573 NotebookApp] Serving notebooks from local directory: /Users/afranzi/Projects/notebooks
[I 19:05:01.573 NotebookApp] The Jupyter Notebook is running at:
[I 19:05:01.573 NotebookApp] http://localhost:8888/?token=c06252daa6a12cfdd33c1d2e96c8d3b19d90e9f6fc171745
[I 19:05:01.573 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 19:05:01.574 NotebookApp]
Copy/paste this URL into your browser when you connect for the first time,
to login with a token:
@afranzi
afranzi / sparkWinePrediction.py
Last active October 23, 2018 22:06
Apply a MLflow model to a Spark Dataframe
import mlflow.pyfunc
model_path = 's3://<bucket>/mlflow/artifacts/1/0f8691808e914d1087cf097a08730f17/artifacts/model'
wine_path = '/Users/afranzi/Projects/data/winequality-red.csv'
wine_udf = mlflow.pyfunc.spark_udf(spark, model_path)
df = spark.read.format("csv").option("header", "true").option('delimiter', ';').load(wine_path)
columns = [ "fixed acidity", "volatile acidity", "citric acid",
"residual sugar", "chlorides", "free sulfur dioxide",
"total sulfur dioxide", "density", "pH",
@afranzi
afranzi / sklearnServe.sh
Last active October 23, 2018 22:06
# Serve a sklearn model through 127.0.0.0:5005
# Serve a sklearn model through 127.0.0.0:5005
MLFLOW_TRACKING_URI=http://0.0.0.0:5000 mlflow sklearn serve \
--port 5005 \
--run_id 0f8691808e914d1087cf097a08730f17 \
--model-path model
@afranzi
afranzi / Wine Quality Prediction - Scala.ipynb
Last active October 25, 2018 21:10
MLflow UDFs from Scala Spark
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.