Skip to content

Instantly share code, notes, and snippets.

@gheesung

gheesung/image-classification-with-sipeed-maix-using-mobilenetv1.ipynb

Created July 18, 2019 08:52
Show Gist options
  • Save gheesung/eb0076e040ba53d5be2ad2db1c70cf82 to your computer and use it in GitHub Desktop.
Save gheesung/eb0076e040ba53d5be2ad2db1c70cf82 to your computer and use it in GitHub Desktop.
Download ZIP
Image Classification With Sipeed Maix using Mobilenetv1
Display the source blob
Display the rendered blob
Raw
image-classification-with-sipeed-maix-using-mobilenetv1.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Image Classification With Sipeed Maix using Mobilenetv1",
"version": "0.3.2",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true,
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/gheesung/eb0076e040ba53d5be2ad2db1c70cf82/image-classification-with-sipeed-maix-using-mobilenetv1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "prOt_VofnYCo",
"colab_type": "text"
},
"source": [
"#Image Classification using Sipeed AIoT.\n",
"\n",
"Sipeed AIoT is an Edge AI MCU which is capable to perform neural network computation at fast speed. In the demo app, Mobilenet V1 model is used and transfer learning is used to train the model to differential between 5 classes of flowers.\n",
"\n",
"This notebook is to combine the steps to create the trained model that can be uploaded to the device.\n",
"\n",
"Acknowledgement: DmitryM8 and the steps are adapted from [Instructables](https://www.instructables.com/id/Transfer-Learning-With-Sipeed-MaiX-and-Arduino-IDE/) with some modifications."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e2hnpFs6P8mC",
"colab_type": "text"
},
"source": [
"## Clone the required files from Github\n",
"\n",
"DmitryM8 version of Mobilenet is used."
]
},
{
"cell_type": "code",
"metadata": {
"id": "zZaQb4KqNBzW",
"colab_type": "code",
"outputId": "a07c4f63-d091-483a-b1a9-d9f04038e684",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 235
}
},
"source": [
"# Clone the libraries to setup the libraries\n",
"\n",
"!git clone https://github.com/AIWintermuteAI/transfer_learning_sipeed.git\n",
"!git clone https://github.com/sipeed/Maix_Toolbox.git"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Cloning into 'transfer_learning_sipeed'...\n",
"remote: Enumerating objects: 59, done.\u001b[K\n",
"remote: Counting objects: 100% (59/59), done.\u001b[K\n",
"remote: Compressing objects: 100% (49/49), done.\u001b[K\n",
"remote: Total 59 (delta 17), reused 25 (delta 5), pack-reused 0\u001b[K\n",
"Unpacking objects: 100% (59/59), done.\n",
"Cloning into 'Maix_Toolbox'...\n",
"remote: Enumerating objects: 34, done.\u001b[K\n",
"remote: Counting objects: 100% (34/34), done.\u001b[K\n",
"remote: Compressing objects: 100% (29/29), done.\u001b[K\n",
"remote: Total 34 (delta 10), reused 16 (delta 4), pack-reused 0\u001b[K\n",
"Unpacking objects: 100% (34/34), done.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vyc9CowltO9q",
"colab_type": "text"
},
"source": [
"### Install the tflite to kmodel conversion software\n",
"Note that there is a bug in the get_nncase.sh in the Maix_Toolbox which is unable to extract the file. The steps below is the same as the script but the typo error is fixed."
]
},
{
"cell_type": "code",
"metadata": {
"id": "nowUGs2NB7Qn",
"colab_type": "code",
"colab": {}
},
"source": [
"%%bash\n",
"cd Maix_Toolbox \n",
"mkdir -p ncc\n",
"mkdir -p workspace\n",
"mkdir -p images\n",
"mkdir -p log\n",
"cd ncc\n",
"wget https://github.com/kendryte/nncase/releases/download/v0.1.0-rc5/ncc-linux-x86_64.tar.xz\n",
"tar -Jxf ncc-linux-x86_64.tar.xz\n",
"rm ncc-linux-x86_64.tar.xz\n",
"echo \"download nncase ok!\""
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "aInHubWZQgzJ",
"colab_type": "text"
},
"source": [
"## Build the model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "4Wvu_zg6NstZ",
"colab_type": "code",
"outputId": "21bfe884-096d-469d-b023-dbd4095dc227",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
}
},
"source": [
"import keras\n",
"import numpy as np\n",
"from keras import backend as K\n",
"from keras.optimizers import Adam\n",
"from keras.metrics import categorical_crossentropy\n",
"from keras.preprocessing.image import ImageDataGenerator\n",
"from keras.preprocessing import image\n",
"from keras.models import Model\n",
"from keras.applications import imagenet_utils\n",
"from keras.layers import Dense, GlobalAveragePooling2D, Dropout,Flatten\n",
"\n",
"\n",
"import sys\n",
"sys.path.append('/content/transfer_learning_sipeed')\n",
"from mobilenet_sipeed.mobilenet import MobileNet\n",
"\n",
"from keras.applications.mobilenet import preprocess_input\n"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"Using TensorFlow backend.\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wLKCaC1cQ8nZ",
"colab_type": "text"
},
"source": [
"### Download the flowers sample dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "t-6gYty7R3u8",
"colab_type": "code",
"outputId": "f8878d00-2411-4297-c35d-546c509c571d",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 90
}
},
"source": [
"# Download the flower photos\n",
"\n",
"%cd /content\n",
"!curl -LO http://download.tensorflow.org/example_images/flower_photos.tgz\n",
"!tar xzf flower_photos.tgz"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
"/content\n",
" % Total % Received % Xferd Average Speed Time Time Time Current\n",
" Dload Upload Total Spent Left Speed\n",
"100 218M 100 218M 0 0 106M 0 0:00:02 0:00:02 --:--:-- 106M\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bumxZ1EFRpd_",
"colab_type": "text"
},
"source": [
"### Define the parameters"
]
},
{
"cell_type": "code",
"metadata": {
"id": "qN8wA-GN3AJK",
"colab_type": "code",
"colab": {}
},
"source": [
"# the parameters\n",
"IMAGE_SIZE = 224\n",
"ALPHA=0.75\n",
"EPOCHS=20"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "wsWFPaOcOyS-",
"colab_type": "code",
"colab": {}
},
"source": [
"def prepare_image(file):\n",
" img_path = ''\n",
" img = image.load_img(img_path + file, target_size=(IMAGE_SIZE, IMAGE_SIZE))\n",
" img_array = image.img_to_array(img)\n",
" img_array_expanded_dims = np.expand_dims(img_array, axis=0)\n",
" return keras.applications.mobilenet.preprocess_input(img_array_expanded_dims)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "wO_tDctxll5i",
"colab_type": "code",
"colab": {}
},
"source": [
"# function to define dropout, hidden layers and the number of output\n",
"def build_finetune_model(base_model, dropout, fc_layers, num_classes):\n",
" for layer in base_model.layers:\n",
" layer.trainable = False\n",
"\n",
" x = base_model.output\n",
" x = GlobalAveragePooling2D()(x)\n",
" \n",
" for fc in fc_layers:\n",
" # New FC layer, random init\n",
" x = Dense(fc, activation='relu')(x) \n",
" x = Dropout(dropout)(x)\n",
"\n",
" # New softmax layer\n",
" predictions = Dense(num_classes, activation='softmax')(x) \n",
" \n",
" finetune_model = Model(inputs=base_model.input, outputs=predictions)\n",
"\n",
" return finetune_model"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "1a6QkUw2RnsH",
"colab_type": "text"
},
"source": [
"### Transfer Learning using Mobilenet V1"
]
},
{
"cell_type": "code",
"metadata": {
"id": "J5oc2DTvO4wf",
"colab_type": "code",
"outputId": "1124dfb5-0fd3-49ba-880c-f99972e84657",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 310
}
},
"source": [
"# Using MobileNetv1\n",
"base_model=MobileNet(input_shape=(IMAGE_SIZE, IMAGE_SIZE, 3), alpha = ALPHA, \n",
" depth_multiplier = 1, dropout = 0.001, include_top = False, \n",
" weights = \"imagenet\", classes = 1000, backend=keras.backend, \n",
" layers=keras.layers,models=keras.models,utils=keras.utils)\n"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"WARNING: Logging before flag parsing goes to stderr.\n",
"W0718 08:15:18.217780 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:74: The name tf.get_default_graph is deprecated. Please use tf.compat.v1.get_default_graph instead.\n",
"\n",
"W0718 08:15:18.267461 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:517: The name tf.placeholder is deprecated. Please use tf.compat.v1.placeholder instead.\n",
"\n",
"W0718 08:15:18.279513 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:4138: The name tf.random_uniform is deprecated. Please use tf.random.uniform instead.\n",
"\n",
"W0718 08:15:18.319509 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:174: The name tf.get_default_session is deprecated. Please use tf.compat.v1.get_default_session instead.\n",
"\n",
"W0718 08:15:18.320745 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:181: The name tf.ConfigProto is deprecated. Please use tf.compat.v1.ConfigProto instead.\n",
"\n",
"W0718 08:15:21.125452 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:1834: The name tf.nn.fused_batch_norm is deprecated. Please use tf.compat.v1.nn.fused_batch_norm instead.\n",
"\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"Downloading data from https://github.com/fchollet/deep-learning-models/releases/download/v0.6/mobilenet_7_5_224_tf_no_top.h5\n",
"10633216/10626956 [==============================] - 2s 0us/step\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ZqKI0E7oRH3h",
"colab_type": "text"
},
"source": [
"### Define the last few layers\n",
"I used 2 hidden layers and 100 and 50 nodes. More layers or nodes can be added but this will increase the model size and may not fit into Maixpy memory."
]
},
{
"cell_type": "code",
"metadata": {
"id": "gzXf_FlDl20y",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 92
},
"outputId": "b58455a9-5eca-4a5a-e7bd-69920f29dd59"
},
"source": [
"FC_LAYERS = [100, 50]\n",
"dropout = 0.5\n",
"\n",
"finetune_model = build_finetune_model(base_model, \n",
" dropout=dropout, \n",
" fc_layers=FC_LAYERS, \n",
" num_classes=5)"
],
"execution_count": 10,
"outputs": [
{
"output_type": "stream",
"text": [
"W0718 08:15:49.377515 140378553849728 deprecation.py:506] From /usr/local/lib/python3.6/dist-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
],
"name": "stderr"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "big9oKAWqQ0N",
"colab_type": "code",
"outputId": "a842203f-9cc9-43c0-86ce-a6432169226a",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
}
},
"source": [
"for i,layer in enumerate(finetune_model.layers):\n",
" print(i,layer.name)"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": [
"0 input_1\n",
"1 conv1_pad\n",
"2 conv1\n",
"3 conv1_bn\n",
"4 conv1_relu\n",
"5 conv_dw_1\n",
"6 conv_dw_1_bn\n",
"7 conv_dw_1_relu\n",
"8 conv_pw_1\n",
"9 conv_pw_1_bn\n",
"10 conv_pw_1_relu\n",
"11 conv_pad_2\n",
"12 conv_dw_2\n",
"13 conv_dw_2_bn\n",
"14 conv_dw_2_relu\n",
"15 conv_pw_2\n",
"16 conv_pw_2_bn\n",
"17 conv_pw_2_relu\n",
"18 conv_dw_3\n",
"19 conv_dw_3_bn\n",
"20 conv_dw_3_relu\n",
"21 conv_pw_3\n",
"22 conv_pw_3_bn\n",
"23 conv_pw_3_relu\n",
"24 conv_pad_4\n",
"25 conv_dw_4\n",
"26 conv_dw_4_bn\n",
"27 conv_dw_4_relu\n",
"28 conv_pw_4\n",
"29 conv_pw_4_bn\n",
"30 conv_pw_4_relu\n",
"31 conv_dw_5\n",
"32 conv_dw_5_bn\n",
"33 conv_dw_5_relu\n",
"34 conv_pw_5\n",
"35 conv_pw_5_bn\n",
"36 conv_pw_5_relu\n",
"37 conv_pad_6\n",
"38 conv_dw_6\n",
"39 conv_dw_6_bn\n",
"40 conv_dw_6_relu\n",
"41 conv_pw_6\n",
"42 conv_pw_6_bn\n",
"43 conv_pw_6_relu\n",
"44 conv_dw_7\n",
"45 conv_dw_7_bn\n",
"46 conv_dw_7_relu\n",
"47 conv_pw_7\n",
"48 conv_pw_7_bn\n",
"49 conv_pw_7_relu\n",
"50 conv_dw_8\n",
"51 conv_dw_8_bn\n",
"52 conv_dw_8_relu\n",
"53 conv_pw_8\n",
"54 conv_pw_8_bn\n",
"55 conv_pw_8_relu\n",
"56 conv_dw_9\n",
"57 conv_dw_9_bn\n",
"58 conv_dw_9_relu\n",
"59 conv_pw_9\n",
"60 conv_pw_9_bn\n",
"61 conv_pw_9_relu\n",
"62 conv_dw_10\n",
"63 conv_dw_10_bn\n",
"64 conv_dw_10_relu\n",
"65 conv_pw_10\n",
"66 conv_pw_10_bn\n",
"67 conv_pw_10_relu\n",
"68 conv_dw_11\n",
"69 conv_dw_11_bn\n",
"70 conv_dw_11_relu\n",
"71 conv_pw_11\n",
"72 conv_pw_11_bn\n",
"73 conv_pw_11_relu\n",
"74 conv_pad_12\n",
"75 conv_dw_12\n",
"76 conv_dw_12_bn\n",
"77 conv_dw_12_relu\n",
"78 conv_pw_12\n",
"79 conv_pw_12_bn\n",
"80 conv_pw_12_relu\n",
"81 conv_dw_13\n",
"82 conv_dw_13_bn\n",
"83 conv_dw_13_relu\n",
"84 conv_pw_13\n",
"85 conv_pw_13_bn\n",
"86 conv_pw_13_relu\n",
"87 global_average_pooling2d_1\n",
"88 dense_1\n",
"89 dropout_1\n",
"90 dense_2\n",
"91 dropout_2\n",
"92 dense_3\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "BayNJNWYSbr0",
"colab_type": "code",
"outputId": "6fa7eb5c-28b8-4243-f153-632cad5e623f",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 35
}
},
"source": [
"train_datagen=ImageDataGenerator(preprocessing_function=preprocess_input) #included in our dependencies\n",
"\n",
"train_generator=train_datagen.flow_from_directory('/content/flower_photos',\n",
" target_size=(IMAGE_SIZE,IMAGE_SIZE),\n",
" color_mode='rgb',\n",
" batch_size=32,\n",
" class_mode='categorical', shuffle=True)"
],
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"text": [
"Found 3670 images belonging to 5 classes.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "PgWO8TPyR7kY",
"colab_type": "text"
},
"source": [
"### Train the Model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "dErWjt8Bn6IV",
"colab_type": "code",
"outputId": "ba77afa7-750c-4650-8519-0b6f73fc0db9",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
}
},
"source": [
"finetune_model.summary()\n",
"finetune_model.compile(optimizer='Adam',loss='categorical_crossentropy',metrics=['accuracy'])\n",
"step_size_train=train_generator.n//train_generator.batch_size\n",
"history = finetune_model.fit_generator(generator=train_generator,steps_per_epoch=step_size_train,epochs=EPOCHS, shuffle=True)\n",
"\n",
"finetune_model.save('/content/my_model.h5')\n",
"\n"
],
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"text": [
"W0718 08:16:11.853531 140378553849728 deprecation_wrapper.py:119] From /usr/local/lib/python3.6/dist-packages/keras/optimizers.py:790: The name tf.train.Optimizer is deprecated. Please use tf.compat.v1.train.Optimizer instead.\n",
"\n",
"W0718 08:16:11.992266 140378553849728 deprecation.py:323] From /usr/local/lib/python3.6/dist-packages/tensorflow/python/ops/math_grad.py:1250: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use tf.where in 2.0, which has the same broadcast rule as np.where\n"
],
"name": "stderr"
},
{
"output_type": "stream",
"text": [
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"input_1 (InputLayer) (None, 224, 224, 3) 0 \n",
"_________________________________________________________________\n",
"conv1_pad (ZeroPadding2D) (None, 226, 226, 3) 0 \n",
"_________________________________________________________________\n",
"conv1 (Conv2D) (None, 112, 112, 24) 648 \n",
"_________________________________________________________________\n",
"conv1_bn (BatchNormalization (None, 112, 112, 24) 96 \n",
"_________________________________________________________________\n",
"conv1_relu (ReLU) (None, 112, 112, 24) 0 \n",
"_________________________________________________________________\n",
"conv_dw_1 (DepthwiseConv2D) (None, 112, 112, 24) 216 \n",
"_________________________________________________________________\n",
"conv_dw_1_bn (BatchNormaliza (None, 112, 112, 24) 96 \n",
"_________________________________________________________________\n",
"conv_dw_1_relu (ReLU) (None, 112, 112, 24) 0 \n",
"_________________________________________________________________\n",
"conv_pw_1 (Conv2D) (None, 112, 112, 48) 1152 \n",
"_________________________________________________________________\n",
"conv_pw_1_bn (BatchNormaliza (None, 112, 112, 48) 192 \n",
"_________________________________________________________________\n",
"conv_pw_1_relu (ReLU) (None, 112, 112, 48) 0 \n",
"_________________________________________________________________\n",
"conv_pad_2 (ZeroPadding2D) (None, 114, 114, 48) 0 \n",
"_________________________________________________________________\n",
"conv_dw_2 (DepthwiseConv2D) (None, 56, 56, 48) 432 \n",
"_________________________________________________________________\n",
"conv_dw_2_bn (BatchNormaliza (None, 56, 56, 48) 192 \n",
"_________________________________________________________________\n",
"conv_dw_2_relu (ReLU) (None, 56, 56, 48) 0 \n",
"_________________________________________________________________\n",
"conv_pw_2 (Conv2D) (None, 56, 56, 96) 4608 \n",
"_________________________________________________________________\n",
"conv_pw_2_bn (BatchNormaliza (None, 56, 56, 96) 384 \n",
"_________________________________________________________________\n",
"conv_pw_2_relu (ReLU) (None, 56, 56, 96) 0 \n",
"_________________________________________________________________\n",
"conv_dw_3 (DepthwiseConv2D) (None, 56, 56, 96) 864 \n",
"_________________________________________________________________\n",
"conv_dw_3_bn (BatchNormaliza (None, 56, 56, 96) 384 \n",
"_________________________________________________________________\n",
"conv_dw_3_relu (ReLU) (None, 56, 56, 96) 0 \n",
"_________________________________________________________________\n",
"conv_pw_3 (Conv2D) (None, 56, 56, 96) 9216 \n",
"_________________________________________________________________\n",
"conv_pw_3_bn (BatchNormaliza (None, 56, 56, 96) 384 \n",
"_________________________________________________________________\n",
"conv_pw_3_relu (ReLU) (None, 56, 56, 96) 0 \n",
"_________________________________________________________________\n",
"conv_pad_4 (ZeroPadding2D) (None, 58, 58, 96) 0 \n",
"_________________________________________________________________\n",
"conv_dw_4 (DepthwiseConv2D) (None, 28, 28, 96) 864 \n",
"_________________________________________________________________\n",
"conv_dw_4_bn (BatchNormaliza (None, 28, 28, 96) 384 \n",
"_________________________________________________________________\n",
"conv_dw_4_relu (ReLU) (None, 28, 28, 96) 0 \n",
"_________________________________________________________________\n",
"conv_pw_4 (Conv2D) (None, 28, 28, 192) 18432 \n",
"_________________________________________________________________\n",
"conv_pw_4_bn (BatchNormaliza (None, 28, 28, 192) 768 \n",
"_________________________________________________________________\n",
"conv_pw_4_relu (ReLU) (None, 28, 28, 192) 0 \n",
"_________________________________________________________________\n",
"conv_dw_5 (DepthwiseConv2D) (None, 28, 28, 192) 1728 \n",
"_________________________________________________________________\n",
"conv_dw_5_bn (BatchNormaliza (None, 28, 28, 192) 768 \n",
"_________________________________________________________________\n",
"conv_dw_5_relu (ReLU) (None, 28, 28, 192) 0 \n",
"_________________________________________________________________\n",
"conv_pw_5 (Conv2D) (None, 28, 28, 192) 36864 \n",
"_________________________________________________________________\n",
"conv_pw_5_bn (BatchNormaliza (None, 28, 28, 192) 768 \n",
"_________________________________________________________________\n",
"conv_pw_5_relu (ReLU) (None, 28, 28, 192) 0 \n",
"_________________________________________________________________\n",
"conv_pad_6 (ZeroPadding2D) (None, 30, 30, 192) 0 \n",
"_________________________________________________________________\n",
"conv_dw_6 (DepthwiseConv2D) (None, 14, 14, 192) 1728 \n",
"_________________________________________________________________\n",
"conv_dw_6_bn (BatchNormaliza (None, 14, 14, 192) 768 \n",
"_________________________________________________________________\n",
"conv_dw_6_relu (ReLU) (None, 14, 14, 192) 0 \n",
"_________________________________________________________________\n",
"conv_pw_6 (Conv2D) (None, 14, 14, 384) 73728 \n",
"_________________________________________________________________\n",
"conv_pw_6_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_pw_6_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_dw_7 (DepthwiseConv2D) (None, 14, 14, 384) 3456 \n",
"_________________________________________________________________\n",
"conv_dw_7_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_dw_7_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pw_7 (Conv2D) (None, 14, 14, 384) 147456 \n",
"_________________________________________________________________\n",
"conv_pw_7_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_pw_7_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_dw_8 (DepthwiseConv2D) (None, 14, 14, 384) 3456 \n",
"_________________________________________________________________\n",
"conv_dw_8_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_dw_8_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pw_8 (Conv2D) (None, 14, 14, 384) 147456 \n",
"_________________________________________________________________\n",
"conv_pw_8_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_pw_8_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_dw_9 (DepthwiseConv2D) (None, 14, 14, 384) 3456 \n",
"_________________________________________________________________\n",
"conv_dw_9_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_dw_9_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pw_9 (Conv2D) (None, 14, 14, 384) 147456 \n",
"_________________________________________________________________\n",
"conv_pw_9_bn (BatchNormaliza (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_pw_9_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_dw_10 (DepthwiseConv2D) (None, 14, 14, 384) 3456 \n",
"_________________________________________________________________\n",
"conv_dw_10_bn (BatchNormaliz (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_dw_10_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pw_10 (Conv2D) (None, 14, 14, 384) 147456 \n",
"_________________________________________________________________\n",
"conv_pw_10_bn (BatchNormaliz (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_pw_10_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_dw_11 (DepthwiseConv2D) (None, 14, 14, 384) 3456 \n",
"_________________________________________________________________\n",
"conv_dw_11_bn (BatchNormaliz (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_dw_11_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pw_11 (Conv2D) (None, 14, 14, 384) 147456 \n",
"_________________________________________________________________\n",
"conv_pw_11_bn (BatchNormaliz (None, 14, 14, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_pw_11_relu (ReLU) (None, 14, 14, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pad_12 (ZeroPadding2D) (None, 16, 16, 384) 0 \n",
"_________________________________________________________________\n",
"conv_dw_12 (DepthwiseConv2D) (None, 7, 7, 384) 3456 \n",
"_________________________________________________________________\n",
"conv_dw_12_bn (BatchNormaliz (None, 7, 7, 384) 1536 \n",
"_________________________________________________________________\n",
"conv_dw_12_relu (ReLU) (None, 7, 7, 384) 0 \n",
"_________________________________________________________________\n",
"conv_pw_12 (Conv2D) (None, 7, 7, 768) 294912 \n",
"_________________________________________________________________\n",
"conv_pw_12_bn (BatchNormaliz (None, 7, 7, 768) 3072 \n",
"_________________________________________________________________\n",
"conv_pw_12_relu (ReLU) (None, 7, 7, 768) 0 \n",
"_________________________________________________________________\n",
"conv_dw_13 (DepthwiseConv2D) (None, 7, 7, 768) 6912 \n",
"_________________________________________________________________\n",
"conv_dw_13_bn (BatchNormaliz (None, 7, 7, 768) 3072 \n",
"_________________________________________________________________\n",
"conv_dw_13_relu (ReLU) (None, 7, 7, 768) 0 \n",
"_________________________________________________________________\n",
"conv_pw_13 (Conv2D) (None, 7, 7, 768) 589824 \n",
"_________________________________________________________________\n",
"conv_pw_13_bn (BatchNormaliz (None, 7, 7, 768) 3072 \n",
"_________________________________________________________________\n",
"conv_pw_13_relu (ReLU) (None, 7, 7, 768) 0 \n",
"_________________________________________________________________\n",
"global_average_pooling2d_1 ( (None, 768) 0 \n",
"_________________________________________________________________\n",
"dense_1 (Dense) (None, 100) 76900 \n",
"_________________________________________________________________\n",
"dropout_1 (Dropout) (None, 100) 0 \n",
"_________________________________________________________________\n",
"dense_2 (Dense) (None, 50) 5050 \n",
"_________________________________________________________________\n",
"dropout_2 (Dropout) (None, 50) 0 \n",
"_________________________________________________________________\n",
"dense_3 (Dense) (None, 5) 255 \n",
"=================================================================\n",
"Total params: 1,915,181\n",
"Trainable params: 82,205\n",
"Non-trainable params: 1,832,976\n",
"_________________________________________________________________\n",
"Epoch 1/20\n",
"114/114 [==============================] - 18s 162ms/step - loss: 1.3878 - acc: 0.4243\n",
"Epoch 2/20\n",
"114/114 [==============================] - 15s 134ms/step - loss: 0.9285 - acc: 0.6390\n",
"Epoch 3/20\n",
"114/114 [==============================] - 15s 130ms/step - loss: 0.7541 - acc: 0.7219\n",
"Epoch 4/20\n",
"114/114 [==============================] - 15s 131ms/step - loss: 0.6676 - acc: 0.7582\n",
"Epoch 5/20\n",
"114/114 [==============================] - 15s 132ms/step - loss: 0.5992 - acc: 0.7921\n",
"Epoch 6/20\n",
"114/114 [==============================] - 15s 132ms/step - loss: 0.5607 - acc: 0.8066\n",
"Epoch 7/20\n",
"114/114 [==============================] - 15s 134ms/step - loss: 0.5140 - acc: 0.8231\n",
"Epoch 8/20\n",
"114/114 [==============================] - 15s 132ms/step - loss: 0.5183 - acc: 0.8245\n",
"Epoch 9/20\n",
"114/114 [==============================] - 15s 134ms/step - loss: 0.5064 - acc: 0.8261\n",
"Epoch 10/20\n",
"114/114 [==============================] - 15s 133ms/step - loss: 0.4582 - acc: 0.8465\n",
"Epoch 11/20\n",
"114/114 [==============================] - 15s 132ms/step - loss: 0.4499 - acc: 0.8469\n",
"Epoch 12/20\n",
"114/114 [==============================] - 15s 130ms/step - loss: 0.4392 - acc: 0.8523\n",
"Epoch 13/20\n",
"114/114 [==============================] - 15s 130ms/step - loss: 0.4308 - acc: 0.8509\n",
"Epoch 14/20\n",
"114/114 [==============================] - 15s 130ms/step - loss: 0.4131 - acc: 0.8620\n",
"Epoch 15/20\n",
"114/114 [==============================] - 15s 132ms/step - loss: 0.4523 - acc: 0.8534\n",
"Epoch 16/20\n",
"114/114 [==============================] - 15s 128ms/step - loss: 0.3946 - acc: 0.8704\n",
"Epoch 17/20\n",
"114/114 [==============================] - 15s 127ms/step - loss: 0.4004 - acc: 0.8653\n",
"Epoch 18/20\n",
"114/114 [==============================] - 15s 129ms/step - loss: 0.3751 - acc: 0.8749\n",
"Epoch 19/20\n",
"114/114 [==============================] - 15s 129ms/step - loss: 0.3665 - acc: 0.8798\n",
"Epoch 20/20\n",
"114/114 [==============================] - 15s 128ms/step - loss: 0.3608 - acc: 0.8768\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "0YBIRTO4u9iM",
"colab_type": "code",
"outputId": "0b1aff84-07ab-4b90-968b-2747345ffccf",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 108
}
},
"source": [
"# do a random test to confirm the model is working\n",
"preprocessed_image = prepare_image('/content/flower_photos/roses/14494590921_3bb1dc7b88_n.jpg')\n",
"predictions_flower = finetune_model.predict(preprocessed_image) \n",
"print(predictions_flower[0][0]*100) \n",
"print(predictions_flower[0][1]*100) \n",
"print(predictions_flower[0][2]*100) \n",
"print(predictions_flower[0][3]*100) \n",
"print(predictions_flower[0][4]*100) "
],
"execution_count": 15,
"outputs": [
{
"output_type": "stream",
"text": [
"4.658864849460542e-07\n",
"8.637944803879805e-08\n",
"97.68275022506714\n",
"1.7272012087232724e-05\n",
"2.317230962216854\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8-4DpV3i_kff",
"colab_type": "text"
},
"source": [
"##Convert the Model\n",
"h5 --> tflite --> kmodel"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8qALZ3-QSWs4",
"colab_type": "text"
},
"source": [
"###h5 --> tflite"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Xh5cMw23YdaE",
"colab_type": "code",
"outputId": "cb3adb47-65b1-449c-a070-57ee2d41dc05",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
}
},
"source": [
"#convert keras to tflite format\n",
"!tflite_convert --output_file=/content/model.tflite --keras_model_file=/content/my_model.h5"
],
"execution_count": 16,
"outputs": [
{
"output_type": "stream",
"text": [
"W0718 08:21:31.884565 140664906188672 deprecation.py:506] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/init_ops.py:1251: calling __init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Call initializer instance with the dtype argument instead of passing it to the constructor\n",
"2019-07-18 08:21:33.419011: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2200000000 Hz\n",
"2019-07-18 08:21:33.419675: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55be958aa380 executing computations on platform Host. Devices:\n",
"2019-07-18 08:21:33.419736: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): <undefined>, <undefined>\n",
"2019-07-18 08:21:33.425571: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcuda.so.1\n",
"2019-07-18 08:21:33.512964: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:33.513731: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55be958aae00 executing computations on platform CUDA. Devices:\n",
"2019-07-18 08:21:33.513783: I tensorflow/compiler/xla/service/service.cc:175] StreamExecutor device (0): Tesla K80, Compute Capability 3.7\n",
"2019-07-18 08:21:33.514032: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:33.514443: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: \n",
"name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
"pciBusID: 0000:00:04.0\n",
"2019-07-18 08:21:33.514737: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0\n",
"2019-07-18 08:21:33.516198: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0\n",
"2019-07-18 08:21:33.525224: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0\n",
"2019-07-18 08:21:33.525718: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0\n",
"2019-07-18 08:21:33.527427: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0\n",
"2019-07-18 08:21:33.534777: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0\n",
"2019-07-18 08:21:33.542835: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7\n",
"2019-07-18 08:21:33.542936: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:33.543290: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:33.543596: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0\n",
"2019-07-18 08:21:33.546034: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0\n",
"2019-07-18 08:21:33.547039: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
"2019-07-18 08:21:33.547078: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 \n",
"2019-07-18 08:21:33.547089: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N \n",
"2019-07-18 08:21:33.549232: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:33.549644: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:33.549977: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:40] Overriding allow_growth setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.\n",
"2019-07-18 08:21:33.550072: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6695 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)\n",
"2019-07-18 08:21:36.404627: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.405024: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1\n",
"2019-07-18 08:21:36.405158: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session\n",
"2019-07-18 08:21:36.405946: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.406266: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: \n",
"name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
"pciBusID: 0000:00:04.0\n",
"2019-07-18 08:21:36.406335: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0\n",
"2019-07-18 08:21:36.406368: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0\n",
"2019-07-18 08:21:36.406404: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0\n",
"2019-07-18 08:21:36.406452: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0\n",
"2019-07-18 08:21:36.406482: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0\n",
"2019-07-18 08:21:36.406527: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0\n",
"2019-07-18 08:21:36.406556: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7\n",
"2019-07-18 08:21:36.406630: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.407023: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.407324: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0\n",
"2019-07-18 08:21:36.407370: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
"2019-07-18 08:21:36.407390: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 \n",
"2019-07-18 08:21:36.407409: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N \n",
"2019-07-18 08:21:36.407742: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.408347: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.408667: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6695 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)\n",
"2019-07-18 08:21:36.443193: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:716] Optimization results for grappler item: graph_to_optimize\n",
"2019-07-18 08:21:36.443231: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] function_optimizer: function_optimizer did nothing. time = 0.002ms.\n",
"2019-07-18 08:21:36.443247: I tensorflow/core/grappler/optimizers/meta_optimizer.cc:718] function_optimizer: function_optimizer did nothing. time = 0ms.\n",
"W0718 08:21:36.461642 140664906188672 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/lite/python/util.py:238: convert_variables_to_constants (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use `tf.compat.v1.graph_util.convert_variables_to_constants`\n",
"W0718 08:21:36.461878 140664906188672 deprecation.py:323] From /usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/graph_util_impl.py:270: extract_sub_graph (from tensorflow.python.framework.graph_util_impl) is deprecated and will be removed in a future version.\n",
"Instructions for updating:\n",
"Use `tf.compat.v1.graph_util.extract_sub_graph`\n",
"I0718 08:21:36.585083 140664906188672 graph_util_impl.py:311] Froze 141 variables.\n",
"I0718 08:21:36.610837 140664906188672 graph_util_impl.py:364] Converted 141 variables to const ops.\n",
"2019-07-18 08:21:36.621104: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.621567: I tensorflow/core/grappler/devices.cc:55] Number of eligible GPUs (core count >= 8, compute capability >= 0.0): 1\n",
"2019-07-18 08:21:36.621661: I tensorflow/core/grappler/clusters/single_machine.cc:359] Starting new session\n",
"2019-07-18 08:21:36.622260: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.622569: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1640] Found device 0 with properties: \n",
"name: Tesla K80 major: 3 minor: 7 memoryClockRate(GHz): 0.8235\n",
"pciBusID: 0000:00:04.0\n",
"2019-07-18 08:21:36.622640: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudart.so.10.0\n",
"2019-07-18 08:21:36.622682: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcublas.so.10.0\n",
"2019-07-18 08:21:36.622731: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcufft.so.10.0\n",
"2019-07-18 08:21:36.622782: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcurand.so.10.0\n",
"2019-07-18 08:21:36.622813: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusolver.so.10.0\n",
"2019-07-18 08:21:36.622841: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcusparse.so.10.0\n",
"2019-07-18 08:21:36.622885: I tensorflow/stream_executor/platform/default/dso_loader.cc:42] Successfully opened dynamic library libcudnn.so.7\n",
"2019-07-18 08:21:36.622956: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.623332: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.623761: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1763] Adding visible gpu devices: 0\n",
"2019-07-18 08:21:36.623824: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1181] Device interconnect StreamExecutor with strength 1 edge matrix:\n",
"2019-07-18 08:21:36.623845: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1187] 0 \n",
"2019-07-18 08:21:36.623864: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1200] 0: N \n",
"2019-07-18 08:21:36.624072: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.624398: I tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:1005] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero\n",
"2019-07-18 08:21:36.624695: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1326] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6695 MB memory) -> physical GPU (device: 0, name: Tesla K80, pci bus id: 0000:00:04.0, compute capability: 3.7)\n",
"2019-07-18 08:21:36.646497: E tensorflow/core/grappler/grappler_item_builder.cc:637] Init node conv1/kernel/Assign doesn't exist in graph\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sHpfWFHgSccx",
"colab_type": "text"
},
"source": [
"### tflite --> kmodel\n",
"\n",
"Prepare some test data for ncc util"
]
},
{
"cell_type": "code",
"metadata": {
"id": "qOO0M8wkORGi",
"colab_type": "code",
"colab": {}
},
"source": [
"%%bash\n",
"cd /content\n",
"\n",
"mkdir /content/test_photos/\n",
"mkdir /content/test_photos/daisy\n",
"mkdir /content/test_photos/dandelion\n",
"mkdir /content/test_photos/roses\n",
"mkdir /content/test_photos/sunflowers\n",
"mkdir /content/test_photos/tulips\n"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "mlXzn2bBO2vF",
"colab_type": "code",
"colab": {}
},
"source": [
"import os\n",
"import shutil\n",
"\n",
"\n",
"def recursive_copy(src, dest):\n",
" \"\"\"\n",
" Copy each file from src dir to dest dir, including sub-directories.\n",
" \"\"\"\n",
" \n",
" #copy only 10 files\n",
" count=0\n",
" for item in os.listdir(src):\n",
" file_path = os.path.join(src, item)\n",
"\n",
" # if item is a file, copy it\n",
" if os.path.isfile(file_path):\n",
" shutil.copy(file_path, dest)\n",
"\n",
" # else if item is a folder, recurse \n",
" elif os.path.isdir(file_path):\n",
" new_dest = os.path.join(dest, item)\n",
" os.mkdir(new_dest)\n",
" recursive_copy(file_path, new_dest)\n",
" if count == 10:\n",
" break\n",
" count+=1\n",
"\n",
"recursive_copy(\"/content/flower_photos/daisy\", \"/content/test_photos/daisy\")\n",
"recursive_copy(\"/content/flower_photos/dandelion\", \"/content/test_photos/dandelion\")\n",
"recursive_copy(\"/content/flower_photos/roses\", \"/content/test_photos/roses\")\n",
"recursive_copy(\"/content/flower_photos/sunflowers\", \"/content/test_photos/sunflowers\")\n",
"recursive_copy(\"/content/flower_photos/tulips\", \"/content/test_photos/tulips\")\n",
"\n"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "bby8_XS8KRBB",
"colab_type": "code",
"outputId": "dc21760e-745d-435e-e162-ecc7bd0859ca",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 837
}
},
"source": [
"#convert tflite to kmodel format\n",
"# this will take some time...\n",
"%cd /content/Maix_Toolbox\n",
"!./ncc/ncc -i tflite -o k210model --dataset /content/test_photos /content/model.tflite /content/model.kmodel"
],
"execution_count": 30,
"outputs": [
{
"output_type": "stream",
"text": [
"/content/Maix_Toolbox\n",
"2019-07-18 08:50:16.655943: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA\n",
"0: InputLayer -> 1x3x224x224\n",
"1: K210Conv2d 1x3x224x224 -> 1x24x112x112\n",
"2: K210Conv2d 1x24x112x112 -> 1x24x112x112\n",
"3: K210Conv2d 1x24x112x112 -> 1x48x112x112\n",
"4: K210Conv2d 1x48x112x112 -> 1x48x112x112\n",
"5: K210Conv2d 1x48x112x112 -> 1x96x56x56\n",
"6: K210Conv2d 1x96x56x56 -> 1x96x56x56\n",
"7: K210Conv2d 1x96x56x56 -> 1x96x56x56\n",
"8: K210Conv2d 1x96x56x56 -> 1x96x56x56\n",
"9: K210Conv2d 1x96x56x56 -> 1x192x28x28\n",
"10: K210Conv2d 1x192x28x28 -> 1x192x28x28\n",
"11: K210Conv2d 1x192x28x28 -> 1x192x28x28\n",
"12: K210Conv2d 1x192x28x28 -> 1x192x28x28\n",
"13: K210Conv2d 1x192x28x28 -> 1x384x14x14\n",
"14: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"15: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"16: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"17: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"18: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"19: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"20: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"21: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"22: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"23: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"24: K210Conv2d 1x384x14x14 -> 1x384x14x14\n",
"25: K210Conv2d 1x384x14x14 -> 1x768x7x7\n",
"26: K210Conv2d 1x768x7x7 -> 1x768x7x7\n",
"27: K210Conv2d 1x768x7x7 -> 1x768x7x7\n",
"28: Dequantize 1x768x7x7 -> 1x768x7x7\n",
"29: GlobalAveragePool 1x768x7x7 -> 1x768x1x1\n",
"30: Reshape 1x768x1x1 -> 1x768\n",
"31: Quantize 1x768 -> 1x768\n",
"32: K210AddPadding 1x768 -> 1x768x4x4\n",
"33: K210Conv2d 1x768x4x4 -> 1x100x4x4\n",
"34: K210Conv2d 1x100x4x4 -> 1x50x4x4\n",
"35: K210Conv2d 1x50x4x4 -> 1x5x4x4\n",
"36: K210RemovePadding 1x5x4x4 -> 1x5\n",
"37: Dequantize 1x5 -> 1x5\n",
"38: Softmax 1x5 -> 1x5\n",
"39: OutputLayer 1x5\n",
"KPU memory usage: 2097152 B\n",
"Main memory usage: 188160 B\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CJ5_9MQQxM_b",
"colab_type": "text"
},
"source": [
"### Download the generated file"
]
},
{
"cell_type": "code",
"metadata": {
"id": "UmYlPtM7xJTV",
"colab_type": "code",
"colab": {}
},
"source": [
"from google.colab import files\n",
"\n",
"files.download('/content/model.kmodel')"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q7az3bQj60Vk",
"colab_type": "text"
},
"source": [
"After this step,\n",
"\n",
"download the kmodel to the \n",
"\n",
"1. Download the kmodel file in /content/model.kmodel\n",
"2. Create a label file. For the flower dataset the content of the label file is\n",
"\n",
"daisy\n",
"\n",
"dandelion\n",
"\n",
"roses\n",
"\n",
"sunflowers\n",
"\n",
"tulips\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "8cfwaYJ4-N8h",
"colab_type": "text"
},
"source": [
"This is the micropython code to be executed on the Sipeed devices. \n",
"(Do not run this in Colab)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "k8L4q7bpnYkj",
"colab_type": "code",
"colab": {}
},
"source": [
"import sensor, image, lcd, time\n",
"import KPU as kpu\n",
"lcd.init()\n",
"sensor.reset()\n",
"sensor.set_pixformat(sensor.RGB565)\n",
"sensor.set_framesize(sensor.QVGA)\n",
"sensor.set_windowing((224, 224))\n",
"sensor.set_vflip(0)\n",
"sensor.set_hmirror(0)\n",
"sensor.run(1)\n",
"lcd.clear()\n",
"lcd.draw_string(100,96,\"MobileNet Demo\")\n",
"lcd.draw_string(100,112,\"Loading labels...\")\n",
"f=open('flowerlabel.txt','r')\n",
"labels=f.readlines()\n",
"f.close()\n",
"task = kpu.load(\"/sd/model.kmodel\") \n",
"clock = time.clock()\n",
"while(True):\n",
" img = sensor.snapshot()\n",
" clock.tick()\n",
" fmap = kpu.forward(task, img)\n",
" fps=clock.fps()\n",
" plist=fmap[:]\n",
" pmax=max(plist)\t\n",
" max_index=plist.index(pmax)\t\n",
" a = lcd.display(img, oft=(50,0))\n",
" lcd.draw_string(0, 224, \"%.2f:%s \"%(pmax, labels[max_index].strip()))\n",
" print(pmax)\n",
"a = kpu.deinit(task)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "KnC5EmFcnpum",
"colab_type": "text"
},
"source": [
"This is the [Youtube Video](https://www.youtube.com/watch?v=0Pc0LwmRWRk) for the realtime inferencing using Sipeed Maixpy Go"
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment