Skip to content

Instantly share code, notes, and snippets.

@jdebecka

jdebecka/ml_numbers_keras.ipynb

Created February 6, 2021 23:04
Show Gist options
  • Save jdebecka/c52d300335ddfa4ba5dfcd04da1a728c to your computer and use it in GitHub Desktop.
Save jdebecka/c52d300335ddfa4ba5dfcd04da1a728c to your computer and use it in GitHub Desktop.
Download ZIP
ML_Numbers_Keras.ipynb
Display the source blob
Display the rendered blob
Raw
ml_numbers_keras.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "ML_Numbers_Keras.ipynb",
"provenance": [],
"authorship_tag": "ABX9TyMiPsafG1IMFXe4Y1SlzXNb",
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/jdebecka/c52d300335ddfa4ba5dfcd04da1a728c/ml_numbers_keras.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "mfK5i2iNBxVK",
"outputId": "806f4553-a717-4fc2-d1a1-c6c97a2108d3"
},
"source": [
"!pip install -U tensorflow_datasets"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting tensorflow_datasets\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/d3/91/c3e36695ca04e6f3c2d920887d7dc36550f6bbb03d7d5fd03c2172b06d97/tensorflow_datasets-4.2.0-py3-none-any.whl (3.7MB)\n",
"\u001b[K |████████████████████████████████| 3.7MB 5.3MB/s \n",
"\u001b[?25hRequirement already satisfied, skipping upgrade: future in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (0.16.0)\n",
"Requirement already satisfied, skipping upgrade: promise in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (2.3)\n",
"Requirement already satisfied, skipping upgrade: requests>=2.19.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (2.23.0)\n",
"Requirement already satisfied, skipping upgrade: attrs>=18.1.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (20.3.0)\n",
"Requirement already satisfied, skipping upgrade: termcolor in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (1.1.0)\n",
"Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (1.15.0)\n",
"Requirement already satisfied, skipping upgrade: dataclasses; python_version < \"3.7\" in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (0.8)\n",
"Requirement already satisfied, skipping upgrade: absl-py in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (0.10.0)\n",
"Requirement already satisfied, skipping upgrade: importlib-resources; python_version < \"3.9\" in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (5.1.0)\n",
"Requirement already satisfied, skipping upgrade: typing-extensions; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (3.7.4.3)\n",
"Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (1.19.5)\n",
"Requirement already satisfied, skipping upgrade: protobuf>=3.12.2 in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (3.12.4)\n",
"Requirement already satisfied, skipping upgrade: tqdm in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (4.41.1)\n",
"Requirement already satisfied, skipping upgrade: tensorflow-metadata in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (0.27.0)\n",
"Requirement already satisfied, skipping upgrade: dill in /usr/local/lib/python3.6/dist-packages (from tensorflow_datasets) (0.3.3)\n",
"Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tensorflow_datasets) (1.24.3)\n",
"Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tensorflow_datasets) (2.10)\n",
"Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tensorflow_datasets) (2020.12.5)\n",
"Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests>=2.19.0->tensorflow_datasets) (3.0.4)\n",
"Requirement already satisfied, skipping upgrade: zipp>=0.4; python_version < \"3.8\" in /usr/local/lib/python3.6/dist-packages (from importlib-resources; python_version < \"3.9\"->tensorflow_datasets) (3.4.0)\n",
"Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.12.2->tensorflow_datasets) (53.0.0)\n",
"Requirement already satisfied, skipping upgrade: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.6/dist-packages (from tensorflow-metadata->tensorflow_datasets) (1.52.0)\n",
"Installing collected packages: tensorflow-datasets\n",
" Found existing installation: tensorflow-datasets 4.0.1\n",
" Uninstalling tensorflow-datasets-4.0.1:\n",
" Successfully uninstalled tensorflow-datasets-4.0.1\n",
"Successfully installed tensorflow-datasets-4.2.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "53YvQmIYB4hm"
},
"source": [
"import tensorflow as tf"
],
"execution_count": 2,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Y2J6dm0EWVpL"
},
"source": [
"keras = tf.keras\n",
"\n",
"(X_train, Y_train), (X_test, Y_test) = keras.datasets.mnist.load_data()"
],
"execution_count": 17,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "j66vvS0hW-f2"
},
"source": [
"## Normalization \n",
"Numbers are represented in form of color pixels ranging from 0 to 255\n",
"\n",
"In order to classify them correctly we will normalize them so they range from 0 to 1\n",
"\n",
"That will sqieeze them and enable us to see correlations better"
]
},
{
"cell_type": "code",
"metadata": {
"id": "ay45Chw4X-zK"
},
"source": [
"X_train = keras.utils.normalize(X_train, axis=1)\n",
"X_test = keras.utils.normalize(X_test, axis=1)"
],
"execution_count": 36,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "LTd_9VqHYIpP"
},
"source": [
"# Model\n",
"\n",
"We define it as a Sequential() which indicates a liniear stack of layers\n",
"\n",
"The layers will be dense because they will be fully connected layer to another layer \n",
"\n",
"Additionally we will flatten our data in order to transla 2d array into 1d array"
]
},
{
"cell_type": "code",
"metadata": {
"id": "aLBr_8TdYE0A"
},
"source": [
"model = keras.models.Sequential()\n",
"\n",
"model.add(keras.layers.Flatten())"
],
"execution_count": 47,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "CK9wIzwuaL8j"
},
"source": [
"## Hidden layer \n",
"\n",
"We will define hidden layers with n neurons and assign ReLu as activation function since it's best fitted to deal with data that used to be multidementional\n",
"\n",
"ReLu outputs a probability distribution abd it;s said that if you don't know what function you should use you would oribably want to start with that one"
]
},
{
"cell_type": "code",
"metadata": {
"id": "pKvfabMQackC"
},
"source": [
"model.add(keras.layers.Dense(128, activation=keras.activations.relu))\n",
"model.add(keras.layers.Dense(128, activation=keras.activations.relu))"
],
"execution_count": 48,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "YD0W9EiTaxWX"
},
"source": [
"## Output layer\n",
"\n",
"This layer will have as many neurons as there are classfiers\n",
"\n",
"In this case we want to classify numbers from 0-10 so we will define 10 neurons \n",
"\n",
"The closing function will be softmax, which is usually used in the closing step of the multi layer network"
]
},
{
"cell_type": "code",
"metadata": {
"id": "LvQ8_2Tea4PE"
},
"source": [
"model.add(keras.layers.Dense(10, activation=keras.activations.softmax))"
],
"execution_count": 49,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "QHMPa3sXbclv"
},
"source": [
"## Compiling model\n",
"\n",
"We will define optimalizer. Again, the default one is adam. However, they are many more that can be used. \n",
"\n",
"loss - will tell us how far we are from the correct answer\n",
"\n",
"metris - defines what parameters we want to track"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "6u2F8-EtccNm",
"outputId": "ffa1f0f1-306c-4f2d-d903-5da513f05517"
},
"source": [
"model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])\n",
"\n",
"model.fit(X_train, Y_train, epochs=5)"
],
"execution_count": 50,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"1875/1875 [==============================] - 6s 3ms/step - loss: 0.4713 - accuracy: 0.8642\n",
"Epoch 2/5\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.1103 - accuracy: 0.9663\n",
"Epoch 3/5\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0713 - accuracy: 0.9773\n",
"Epoch 4/5\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0502 - accuracy: 0.9842\n",
"Epoch 5/5\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0384 - accuracy: 0.9876\n"
],
"name": "stdout"
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f8e627fb7f0>"
]
},
"metadata": {
"tags": []
},
"execution_count": 50
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "2bpdVAO7cktc"
},
"source": [
""
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "I1-gQm41fPQu"
},
"source": [
"# Model evaluation \n",
"\n",
"Models can be easily overfitted in order to prevent that we will calculate validation loss and accuracy \n",
"\n",
"The loss and accuracy of thest samples will vary from the train ones. \n",
"\n",
"We have to be caureful to not to be too close nor too far because that would indicate either overfitting or underfitting"
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "gJO8buk9f7B0",
"outputId": "f43f8fd7-e8d7-4bc9-fbb0-64c7602b7a01"
},
"source": [
"validation_loss, validation_acc = model.evaluate(X_test, Y_test)\n"
],
"execution_count": 51,
"outputs": [
{
"output_type": "stream",
"text": [
"313/313 [==============================] - 1s 1ms/step - loss: 0.0872 - accuracy: 0.9749\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "zgrTqDIrgEZz",
"outputId": "b480b01f-99d3-47b2-ff4f-142804032d73"
},
"source": [
"model.fit(X_train, Y_train, epochs=50)\n"
],
"execution_count": 54,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0025 - accuracy: 0.9993\n",
"Epoch 2/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0035 - accuracy: 0.9992\n",
"Epoch 3/50\n",
"1875/1875 [==============================] - 6s 3ms/step - loss: 0.0019 - accuracy: 0.9997\n",
"Epoch 4/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0054 - accuracy: 0.9988\n",
"Epoch 5/50\n",
"1875/1875 [==============================] - 6s 3ms/step - loss: 0.0036 - accuracy: 0.9993\n",
"Epoch 6/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0016 - accuracy: 0.9996\n",
"Epoch 7/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0030 - accuracy: 0.9993\n",
"Epoch 8/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0027 - accuracy: 0.9995\n",
"Epoch 9/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0052 - accuracy: 0.9987\n",
"Epoch 10/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0027 - accuracy: 0.9992\n",
"Epoch 11/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0035 - accuracy: 0.9991\n",
"Epoch 12/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0034 - accuracy: 0.9992\n",
"Epoch 13/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0025 - accuracy: 0.9994\n",
"Epoch 14/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 8.5316e-04 - accuracy: 0.9998\n",
"Epoch 15/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0044 - accuracy: 0.9991\n",
"Epoch 16/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0027 - accuracy: 0.9994\n",
"Epoch 17/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0019 - accuracy: 0.9995\n",
"Epoch 18/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0025 - accuracy: 0.9993\n",
"Epoch 19/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0035 - accuracy: 0.9992\n",
"Epoch 20/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0024 - accuracy: 0.9995\n",
"Epoch 21/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0042 - accuracy: 0.9992\n",
"Epoch 22/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0024 - accuracy: 0.9993\n",
"Epoch 23/50\n",
"1875/1875 [==============================] - 6s 3ms/step - loss: 0.0028 - accuracy: 0.9993\n",
"Epoch 24/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0028 - accuracy: 0.9993\n",
"Epoch 25/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0031 - accuracy: 0.9992\n",
"Epoch 26/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0031 - accuracy: 0.9995\n",
"Epoch 27/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0032 - accuracy: 0.9992\n",
"Epoch 28/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0023 - accuracy: 0.9997\n",
"Epoch 29/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0034 - accuracy: 0.9994\n",
"Epoch 30/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0032 - accuracy: 0.9994\n",
"Epoch 31/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0022 - accuracy: 0.9995\n",
"Epoch 32/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0014 - accuracy: 0.9996\n",
"Epoch 33/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0023 - accuracy: 0.9995\n",
"Epoch 34/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0040 - accuracy: 0.9992\n",
"Epoch 35/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0032 - accuracy: 0.9995\n",
"Epoch 36/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0025 - accuracy: 0.9995\n",
"Epoch 37/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0042 - accuracy: 0.9992\n",
"Epoch 38/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0022 - accuracy: 0.9995\n",
"Epoch 39/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0022 - accuracy: 0.9995\n",
"Epoch 40/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0032 - accuracy: 0.9994\n",
"Epoch 41/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0020 - accuracy: 0.9996\n",
"Epoch 42/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0022 - accuracy: 0.9996\n",
"Epoch 43/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0034 - accuracy: 0.9993\n",
"Epoch 44/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0034 - accuracy: 0.9994\n",
"Epoch 45/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 6.1906e-04 - accuracy: 0.9998\n",
"Epoch 46/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0065 - accuracy: 0.9989\n",
"Epoch 47/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0035 - accuracy: 0.9994\n",
"Epoch 48/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0024 - accuracy: 0.9995\n",
"Epoch 49/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0024 - accuracy: 0.9995\n",
"Epoch 50/50\n",
"1875/1875 [==============================] - 5s 3ms/step - loss: 0.0030 - accuracy: 0.9994\n"
],
"name": "stdout"
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7f8e5f4632b0>"
]
},
"metadata": {
"tags": []
},
"execution_count": 54
}
]
},
{
"cell_type": "code",
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "cl9qimuUkCvv",
"outputId": "1e0f2013-69fe-4026-c037-e41f41c9b962"
},
"source": [
"validation_loss, validation_acc = model.evaluate(X_test, Y_test)\n"
],
"execution_count": 56,
"outputs": [
{
"output_type": "stream",
"text": [
"313/313 [==============================] - 0s 1ms/step - loss: 0.3983 - accuracy: 0.9803\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RO_-9dOFkNn8"
},
"source": [
"# Overfitting\n",
"\n",
"After 50 epoch the modedl insead of learnign how to classify memorized the pictures. \n",
"\n",
"We can conclude that by the big diffrence between train & test loss and accuracy "
]
},
{
"cell_type": "code",
"metadata": {
"id": "cqEiNpmoggQ8"
},
"source": [
""
],
"execution_count": null,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment