Image-Classification-catdog-MLOps.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Keras による画像認識モデル開発と自動パラメータチューニング (Hyperdrive)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Keras を利用したモデル学習を行います。犬、猫の画像を区別する画像認識モデルを構築します。後半には、[Hyperdrive](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-tune-hyperparameters) によるハイパーパラメータチューニングによって精度が高いモデルを探索します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. 事前準備\n",
    "    - Python SDK のインポート\n",
    "    - Azure ML `Workspace` への接続\n",
    "    - `Experiment` の作成\n",
    "    - `Dataset` の作成と登録\n",
    "2. Keras による画像認識モデルの開発\n",
    "    - 画像認識モデルの構築\n",
    "    - `Model` の登録と `Dataset` の紐付け\n",
    "3. 自動パラメータチューニング Hyperdrvie\n",
    "    - 計算環境 `Machine Learning Compute` の準備\n",
    "    - 自動パラメータチューニング `Hyperdrive` の事前設定\n",
    "    - モデル学習と結果の確認"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. 事前準備"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python SDK インポート\n",
    "Azure Machine Learning python SDK をインポートします。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-info\"><h4> Python 環境について</h4><p>\n",
    "    \n",
    "Azure Machine Learning Python SDK が導入されている環境であれば、オンプレミスのクライアント PC の Python 環境からもご利用になれます。クライアント PC へのインストール方法については、ドキュメントをご参照ください。\n",
    "    \n",
    "**ドキュメント**：[Azure Machine Learning のための開発環境を構成する](https://docs.microsoft.com/ja-jp/azure/machine-learning/how-to-configure-environment#local)\n",
    "\n",
    "</p></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace, Dataset, Experiment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Azure Machine Learning Python SDK のバージョンを確認します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(azureml.core.VERSION)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Azure Machine Learning Workspace への接続"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "ws = Workspace.from_config()\n",
    "print(ws.name, ws.location, ws.resource_group, ws.location, sep = '\\t')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # tenant id を指定する場合\n",
    "# from azureml.core.authentication import InteractiveLoginAuthentication\n",
    "# interactive_auth = InteractiveLoginAuthentication(tenant_id=\"72f988bf-86f1-41af-91ab-2d7cd011db47\")\n",
    "# ws = Workspace.from_config(auth=interactive_auth)\n",
    "# print(ws.name, ws.location, ws.resource_group, ws.location, sep = '\\t')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 実験名の設定\n",
    "Azure Machine Learing では 実験を管理する仕組みがあります。自動機械学習は自動的にその実験管理の仕組みでメトリックやログが残ります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "experiment = Experiment(workspace = ws, name = \"keras_catdog_hyperdrive\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 学習データの準備"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Azure Machine Learning service の計算環境 Compute Cluster (コンピューティングクラスター) におけるモデル学習でクラウド上のデータを参照するようにするため、Azure Machine Learning Dataset を作成します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "dogcat = Dataset.get_by_name(ws, name='dogcat')\n",
    "print(\"Name:\", dogcat.name, \", version:\",dogcat.version)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#dogcat.to_path()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 計算環境 (Machine Learning Compute) の設定\n",
    "Azure Machine Learning では機械学習のための計算環境 Machine Learning Compute が利用できます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-warning\"><h4> 計算環境について</h4><p>\n",
    "\n",
    "GPU のクォータがある場合は、新たに GPU の計算環境を構築して利用することを推奨します。\n",
    "\n",
    "※ 参考ドキュメント：[Azure リソースのクォータの管理と要求](https://docs.microsoft.com/ja-jp/azure/machine-learning/how-to-manage-quotas)\n",
    "    \n",
    "</p></div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 予め Azure ML studio にて gpu-cluster という名称の Machine Learning Compute を 作成しておく\n",
    "from azureml.core.compute import ComputeTarget\n",
    "compute_target = ComputeTarget(ws,\"cpucluster\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### モデル学習 Pythonスクリプト"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "project_folder=\"./keras_hyperdrive\"\n",
    "os.makedirs(project_folder, exist_ok=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile {project_folder}/keras_dogcat.py\n",
    "\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "\n",
    "from PIL import Image\n",
    "from tensorflow.python.keras.preprocessing.image import ImageDataGenerator\n",
    "from tensorflow.python.keras.preprocessing.image import array_to_img\n",
    "from tensorflow.python.keras.preprocessing.image import img_to_array\n",
    "from tensorflow.python.keras.preprocessing.image import load_img\n",
    "from tensorflow.python.keras.layers import Conv2D, MaxPooling2D\n",
    "from tensorflow.python.keras.layers import Activation, Dropout, Flatten, Input, Dense\n",
    "from tensorflow.python.keras.models import Model\n",
    "import os\n",
    "from azureml.core import Run\n",
    "from azureml.core import Workspace, Dataset\n",
    "from keras.utils import plot_model\n",
    "import argparse\n",
    "\n",
    "print(\"## START Script ##\")\n",
    "\n",
    "\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--batch-size', type=int, dest='batch_size', default=16, help='mini batch size for training')\n",
    "parser.add_argument('--learning-rate', type=float, dest='learning_rate', default=0.01, help='learning rate')\n",
    "parser.add_argument('--dropout', type=float, dest='dropout', default=0.01, help='dropout rate')\n",
    "parser.add_argument('--epoch', type=int, dest='epoch', default=10, help='number of epoch')\n",
    "\n",
    "\n",
    "args = parser.parse_args()\n",
    "\n",
    "\n",
    "\n",
    "batch_size = args.batch_size\n",
    "learing_rate = args.learning_rate\n",
    "dropout = args.dropout\n",
    "epoch = args.epoch\n",
    "\n",
    "run = Run.get_context()\n",
    "\n",
    "data_folder=\"./data\"\n",
    "os.makedirs(data_folder, exist_ok=True)\n",
    "\n",
    "\n",
    "\n",
    "# Tempフォルダにデータをダウンロード (or マウント)\n",
    "# 参考: https://docs.microsoft.com/ja-jp/azure/machine-learning/how-to-train-with-datasets\n",
    "import tempfile\n",
    "mounted_path = tempfile.mkdtemp()\n",
    "print(\"mount path\", mounted_path)\n",
    "\n",
    "# dogcat データセットを指定\n",
    "mount_context = run.input_datasets['dogcat'].download(mounted_path)\n",
    "\n",
    "\n",
    "## ダウンロードではなくマウントした場合は下記を実行。mount_context.start() と mount_context.stop() も必要。\n",
    "#mount_context = run.input_datasets['dogcat'].mount(mounted_path)\n",
    "#mount_context.start()\n",
    "\n",
    "print(os.listdir(mounted_path))\n",
    "print(mounted_path)\n",
    "\n",
    "\n",
    "class RunCallback(tf.keras.callbacks.Callback):\n",
    "    def __init__(self, run):\n",
    "        self.run = run\n",
    "        \n",
    "    def on_epoch_end(self, batch, logs={}):\n",
    "        self.run.log(name=\"training_acc\", value=float(logs.get('acc')))\n",
    "        self.run.log(name=\"validation_acc\", value=float(logs.get('val_acc')))\n",
    "    \n",
    "\n",
    "inputs = Input(shape=(150, 150, 3))\n",
    "x = Conv2D(32, (3, 3))(inputs)\n",
    "x = Activation(\"relu\")(x)\n",
    "x = MaxPooling2D(pool_size=(2, 2))(x)\n",
    "\n",
    "x = Conv2D(32, (3, 3))(x)\n",
    "x = Activation(\"relu\")(x)\n",
    "x = MaxPooling2D(pool_size=(2, 2))(x)\n",
    "\n",
    "x = Conv2D(64, (3, 3))(x)\n",
    "x = Activation(\"relu\")(x)\n",
    "x = MaxPooling2D(pool_size=(2, 2))(x)\n",
    "\n",
    "x = Flatten()(x)\n",
    "x = Dense(64, activation=\"relu\")(x)\n",
    "x = Dropout(dropout)(x)\n",
    "prediction = Dense(1, activation=\"sigmoid\")(x)\n",
    "\n",
    "model = Model(inputs=inputs, outputs=prediction)\n",
    "model.compile(loss=\"binary_crossentropy\",optimizer=\"rmsprop\",metrics=[\"accuracy\"])\n",
    "\n",
    "\n",
    "callbacks = list()\n",
    "callbacks.append(RunCallback(run))\n",
    "\n",
    "train_datagen = ImageDataGenerator(\n",
    "    rescale=1./255,\n",
    "    shear_range=0.2,\n",
    "    zoom_range=0.2,\n",
    "    horizontal_flip=True,\n",
    "    validation_split=0.2)\n",
    "\n",
    "\n",
    "train_generator = train_datagen.flow_from_directory(\n",
    "    mounted_path,\n",
    "    target_size=(150, 150), # resize\n",
    "    batch_size=batch_size,\n",
    "    class_mode=\"binary\",\n",
    "    subset='training')\n",
    "\n",
    "validation_generator = train_datagen.flow_from_directory(\n",
    "    mounted_path,\n",
    "    target_size=(150, 150),\n",
    "    batch_size=batch_size,\n",
    "    class_mode=\"binary\",\n",
    "    subset='validation')\n",
    "\n",
    "VERBOSE = 1\n",
    "\n",
    "print(\"## START TRAINING ##\")\n",
    "\n",
    "run.log(\"Batch size\",batch_size)\n",
    "run.log(\"Num of Epoch\", epoch)\n",
    "run.log(\"Dropout Rate\", dropout)\n",
    "\n",
    "\n",
    "model.fit_generator(\n",
    "    train_generator,\n",
    "    steps_per_epoch=train_generator.samples // batch_size,\n",
    "    epochs=epoch,\n",
    "    validation_data=validation_generator,\n",
    "    validation_steps=validation_generator.samples // batch_size,\n",
    "    callbacks= callbacks)\n",
    "\n",
    "filename = \"keras_simple.h5\"\n",
    "model.save_weights(\"outputs/\"+filename)\n",
    "run.upload_file(\"outputs/\"+filename, \"outputs/\"+filename)\n",
    "\n",
    "\n",
    "# モデル登録。dogcat データセットを学習に利用したことを設定。\n",
    "run.register_model(model_name='keras-dogcat',\n",
    "                   tags={'data': 'dog and cat image'},\n",
    "                   model_path='outputs/keras_simple.h5', \n",
    "                   datasets=[(\"train\",run.input_datasets['dogcat'])])\n",
    "\n",
    "#mount_context.stop()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Keras による画像認識モデルの開発"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### TensorFlow Estimator 設定\n",
    "Keras(TensorFlow backend) を用いたモデル学習の事前設定を行います。[TensorFlow Estimator](https://docs.microsoft.com/ja-JP/python/api/azureml-train-core/azureml.train.dnn.tensorflow) を用いて定義します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.dnn import TensorFlow\n",
    "\n",
    "script_params = {\n",
    "    '--batch-size': 64,\n",
    "    '--learning-rate': 0.01,\n",
    "    '--dropout': 0.03,\n",
    "    '--epoch':1\n",
    "}\n",
    "\n",
    "\n",
    "estimator = TensorFlow(source_directory=project_folder,\n",
    "                       script_params = script_params,\n",
    "                       compute_target=compute_target,\n",
    "                       entry_script='keras_dogcat.py',\n",
    "                       framework_version = '1.13',\n",
    "                       pip_packages=['keras','Pillow'],\n",
    "                       inputs=[dogcat.as_named_input('dogcat')]\n",
    "                      )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### モデル学習と結果確認\n",
    "一番最初の実行は Docker Image を作成するため、20 〜 30分ほど時間がかかります。<br>Dokcer Image を Build している様子は Azure Machine Learning studio から確認できます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "run = experiment.submit(estimator)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run.get_portal_url()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from azureml.widgets import RunDetails\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # 詳細ログの出力\n",
    "# run.get_details()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Machine Learning Pipeline による再利用可能なパイプライン構築\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上記で作成したモデル学習のプロセスを Pipeline として、再実行可能なパイプラインとして構築して実行します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.pipeline.steps import EstimatorStep"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.dnn import TensorFlow\n",
    "\n",
    "est_pipeline = TensorFlow(source_directory=project_folder, \n",
    "                compute_target=compute_target, \n",
    "                entry_script='keras_dogcat.py', \n",
    "                framework_version = '1.13',\n",
    "                pip_packages = ['keras','Pillow'],\n",
    "               )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.pipeline.steps import EstimatorStep\n",
    "\n",
    "est_step = EstimatorStep(name=\"DogCat_Model_Train\",\n",
    "                         estimator=est_pipeline, \n",
    "                         estimator_entry_script_arguments = [],\n",
    "                         runconfig_pipeline_params=None, \n",
    "                         inputs=[dogcat.as_named_input('dogcat')], \n",
    "                         compute_target=compute_target)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.pipeline.core import Pipeline\n",
    "from azureml.core import Experiment\n",
    "pipeline = Pipeline(workspace=ws, steps=[est_step])\n",
    "pipeline_run = Experiment(ws, 'DogCat_Pipeline').submit(pipeline)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.widgets import RunDetails\n",
    "RunDetails(pipeline_run).show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 詳細ログの出力\n",
    "run.get_details()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. 自動パラメータチューニング Hyperdrvie"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Hyperdrive ハイパーパラメータチューニング\n",
    "自動パラメータチューニングの事前設定を行います。[HyperDriveConfig](https://docs.microsoft.com/ja-jp/python/api/azureml-train-core/azureml.train.hyperdrive.hyperdriveconfig?view=azure-ml-py) を用いて定義します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.dnn import TensorFlow\n",
    "\n",
    "tf_hyperdrive_est = TensorFlow(source_directory=project_folder,\n",
    "                       #script_params = script_params,\n",
    "                       compute_target=compute_target,\n",
    "                       entry_script='keras_dogcat.py',\n",
    "                       framework_version = '1.13',\n",
    "                       pip_packages=['keras','Pillow'],\n",
    "                       inputs=[dogcat.as_named_input('dogcat')]\n",
    "                      )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "HyperDrive が探索するパラメータの範囲を定義します。\n",
    "\n",
    "※ 詳細は  [Azure Machine Learning でモデルのハイパーパラメーターを調整する](https://docs.microsoft.com/ja-jp/azure/machine-learning/service/how-to-tune-hyperparameters) をご参照ください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.hyperdrive import *\n",
    "\n",
    "ps = RandomParameterSampling(\n",
    "    {\n",
    "        '--batch-size': choice(25, 50, 75,100),\n",
    "        '--learning-rate': loguniform(-6, -1),\n",
    "        '--dropout': loguniform(-6, -1),\n",
    "        '--epoch' : choice(range(1,10))\n",
    "    }\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 早期停止ポリシー (2イテレーション毎チェック、トップ10%未満の場合は停止)\n",
    "policy = BanditPolicy(evaluation_interval=2, slack_factor=0.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "htc = HyperDriveConfig(estimator=tf_hyperdrive_est, \n",
    "                          hyperparameter_sampling=ps, \n",
    "                          policy=policy, \n",
    "                          primary_metric_name='validation_acc', \n",
    "                          primary_metric_goal=PrimaryMetricGoal.MAXIMIZE, \n",
    "                          max_total_runs=8,  # トータル試行回数\n",
    "                          max_concurrent_runs=4)  # 最大並列度"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Hyperdrive によるモデル学習と結果確認"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "htr = experiment.submit(config=htc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "RunDetails(htr).show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# # 詳細ログの出力\n",
    "# htr.get_details()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# HyperDrive によるモデル学習完了後に実行すること\n",
    "best_run = htr.get_best_run_by_primary_metric()\n",
    "best_run_metrics = best_run.get_metrics()\n",
    "print(best_run)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "run"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernel_info": {
   "name": "python3-azureml"
  },
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  },
  "nteract": {
   "version": "nteract-front-end@1.0.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}