Skip to content

Instantly share code, notes, and snippets.

@ia35

ia35/iris_tf_2x.ipynb

Created April 24, 2020 13:19
Show Gist options
  • Save ia35/239b02f696f55ae6edff5aaf77f6d337 to your computer and use it in GitHub Desktop.
Save ia35/239b02f696f55ae6edff5aaf77f6d337 to your computer and use it in GitHub Desktop.
Download ZIP
iris_tf_2x.ipynb
Display the source blob
Display the rendered blob
Raw
iris_tf_2x.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "iris_tf_2x.ipynb",
"provenance": [],
"private_outputs": true,
"collapsed_sections": [],
"include_colab_link": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "view-in-github",
"colab_type": "text"
},
"source": [
"<a href=\"https://colab.research.google.com/gist/ia35/239b02f696f55ae6edff5aaf77f6d337/iris_tf_2x.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4UJ0pipE1lIP",
"colab_type": "text"
},
"source": [
"[![](http://bec552ebfe.url-de-test.ws/ml/buttonBackProp.png)](https://www.backprop.fr)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SUcb4Aw91qHV",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">\n",
"Le logo BackProp est présenté chaque fois qu'une modification importante est apportée au code ou à chaque fois qu'un commentaire doit être signalé. \n",
"\n",
"Le texte de cette couleur est soit le nôtre, en français, soit un extrait de site qui apporte des explications.\n",
"</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uq13sE-R2Iag",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">Il y a 2 ans, une éternité, un [tutoriel](https://intelligence-artificielle.agency/tensorflow-tutoriel-1/) de classification d'iris, à l'aide de TensorFlow 1.x avait été présenté.\n",
"\n",
"<font color=\"LightSlateGray\">Depuis cette date, TensorFlow a fondamentalement changé. La version 2.x est très différente de la 1.x. Raison pour laquelle, l'article précédent est obsolète.\n",
"\n",
"<font color=\"LightSlateGray\">Le nouveau tutoriel de classication d'iris disponible sur le site de TensorFlow est repris ici et commenté."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "v2ZrKihQ19Ub",
"colab_type": "text"
},
"source": [
"# <font color=\"teal\">Références</font>\n",
"- Custom training: [walkthrough](https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough)\n",
"- Premade [Estimators](https://www.tensorflow.org/tutorials/estimator/premade)\n",
"- TensorFlow – [tutoriel #1](https://intelligence-artificielle.agency/tensorflow-tutoriel-1/)\n",
"- Tutorial [Classification](https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/tutorials/tut2.pdf)\n",
"- [Scatter](https://benalexkeen.com/scatter-matrices-using-pandas/) Matrices using pandas\n",
"- How to use Pandas [Scatter Matrix](https://www.marsja.se/pandas-scatter-matrix-pair-plot/)\n",
"- [Dataset](http://tensorflow.backprop.fr/dataset-5-fleurs/) 5 fleurs\n",
"- How does the [Softmax](https://www.machinecurve.com/index.php/2020/01/08/how-does-the-softmax-activation-function-work/) activation function work?"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "rwxGnsA92emp"
},
"source": [
"##### Copyright 2018 The TensorFlow Authors."
]
},
{
"cell_type": "code",
"metadata": {
"cellView": "both",
"colab_type": "code",
"id": "CPII1rGR2rF9",
"colab": {}
},
"source": [
"#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "DUCMEhba3yXj",
"colab_type": "text"
},
"source": [
"# <font color=\"teal\">Introduction</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "phZJsL_3397N",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">TensorFlow est une plate-forme logicielle permettant de créer des modèles de machine learning (ML).\n",
"\n",
"<font color=\"LightSlateGray\">Si vous êtes débutant avec TensorFlow, commencez par lire **TensorFlow 2** [**quickstart**](https://www.tensorflow.org/tutorials/quickstart/beginner) **for beginners** sur le site officiel de TensorFlow puis [For beginners](https://www.tensorflow.org/tutorials). Ce quickstart est un peu le Hello World du DL (MNIST). "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OUuzz8ct872V",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">Des commentaires ont été ajoutés au code officiel. En effet sauf si le DL (Deep Learning) n’a plus de secrets pour vous et que vous programmez chaque jour en Python avec NumPy, Pandas, … alors il risque d’y avoir quelques lignes difficilement compréhensibles."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "JtEZ1pCPn--z"
},
"source": [
"# Custom training: walkthrough"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "GV1F7tVTN3Dn"
},
"source": [
"<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://www.tensorflow.org/tutorials/customization/custom_training_walkthrough\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/tutorials/customization/custom_training_walkthrough.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/tutorials/customization/custom_training_walkthrough.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" />View source on GitHub</a>\n",
" </td>\n",
" <td>\n",
" <a href=\"https://storage.googleapis.com/tensorflow_docs/docs/site/en/tutorials/customization/custom_training_walkthrough.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "LDrzLFXE8T1l"
},
"source": [
"This guide uses machine learning to *categorize* Iris flowers by species. It uses TensorFlow to:\n",
"1. Build a model,\n",
"2. Train this model on example data, and\n",
"3. Use the model to make predictions about unknown data.\n",
"\n",
"## TensorFlow programming\n",
"\n",
"This guide uses these high-level TensorFlow concepts:\n",
"\n",
"* Use TensorFlow's default [eager execution](../../guide/eager.ipynb) development environment,\n",
"* Import data with the [Datasets API](../../guide/datasets.ipynb),\n",
"* Build models and layers with TensorFlow's [Keras API](../../guide/keras/overview.ipynb).\n",
"\n",
"This tutorial is structured like many TensorFlow programs:\n",
"\n",
"1. Import and parse the dataset.\n",
"2. Select the type of model.\n",
"3. Train the model.\n",
"4. Evaluate the model's effectiveness.\n",
"5. Use the trained model to make predictions."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "yNr7H-AIoLOR"
},
"source": [
"## Setup program"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "1J3AuPBT9gyR"
},
"source": [
"### Configure imports\n",
"\n",
"Import TensorFlow and the other required Python modules. By default, TensorFlow uses [eager execution](../../guide/eager.ipynb) to evaluate operations immediately, returning concrete values instead of creating a computational graph that is executed later. If you are used to a REPL or the `python` interactive console, this feels familiar."
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "jElLULrDhQZR",
"colab": {}
},
"source": [
"import os\n",
"import matplotlib.pyplot as plt"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "bfV2Dai0Ow2o",
"colab": {}
},
"source": [
"import tensorflow as tf"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "g4Wzg69bnwK2",
"colab": {}
},
"source": [
"print(\"TensorFlow version: {}\".format(tf.__version__))\n",
"print(\"Eager execution: {}\".format(tf.executing_eagerly()))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "wyIGHBPpBQnH",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">\n",
"Le test pour savoir si on est en mode eager est inutile car depuis la version 2.x de Tf on est obligatoirement en mode eager.\n",
"</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Zx7wc0LuuxaJ"
},
"source": [
"## The Iris classification problem\n",
"\n",
"Imagine you are a botanist seeking an automated way to categorize each Iris flower you find. Machine learning provides many algorithms to classify flowers statistically. For instance, a sophisticated machine learning program could classify flowers based on photographs. Our ambitions are more modest—we're going to classify Iris flowers based on the length and width measurements of their [sepals](https://en.wikipedia.org/wiki/Sepal) and [petals](https://en.wikipedia.org/wiki/Petal).\n",
"\n",
"The Iris genus entails about 300 species, but our program will only classify the following three:\n",
"\n",
"* Iris setosa\n",
"* Iris virginica\n",
"* Iris versicolor\n",
"\n",
"<table>\n",
" <tr><td>\n",
" <img src=\"https://www.tensorflow.org/images/iris_three_species.jpg\"\n",
" alt=\"Petal geometry compared for three iris species: Iris setosa, Iris virginica, and Iris versicolor\">\n",
" </td></tr>\n",
" <tr><td align=\"center\">\n",
" <b>Figure 1.</b> <a href=\"https://commons.wikimedia.org/w/index.php?curid=170298\">Iris setosa</a> (by <a href=\"https://commons.wikimedia.org/wiki/User:Radomil\">Radomil</a>, CC BY-SA 3.0), <a href=\"https://commons.wikimedia.org/w/index.php?curid=248095\">Iris versicolor</a>, (by <a href=\"https://commons.wikimedia.org/wiki/User:Dlanglois\">Dlanglois</a>, CC BY-SA 3.0), and <a href=\"https://www.flickr.com/photos/33397993@N05/3352169862\">Iris virginica</a> (by <a href=\"https://www.flickr.com/photos/33397993@N05\">Frank Mayfield</a>, CC BY-SA 2.0).<br/>&nbsp;\n",
" </td></tr>\n",
"</table>\n",
"\n",
"Fortunately, someone has already created a [dataset of 120 Iris flowers](https://en.wikipedia.org/wiki/Iris_flower_data_set) with the sepal and petal measurements. This is a classic dataset that is popular for beginner machine learning classification problems."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "I2r8GtyMCIz1",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">\n",
"Les données qui servent dans cet exemple sont celles du fichier iris visible sur Wikipedia.\n",
"Trois espèces d'iris dans la base : Iris setosa, Iris virginica et Iris versicolor. Quatre caractéristiques ont été mesurées à partir de chaque échantillon : la longueur et la largeur des sépales et des pétales, en centimètres."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "T0l8aJpXCpSx",
"colab_type": "text"
},
"source": [
"<img class=\"alignnone size-full wp-image-588\" src=\"http://intelligence-artificielle.agency/wp-content/uploads/2018/03/fleur_schema_coupe.gif\" alt=\"\" width=\"647\" height=\"326\" />"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "3Px6KAg0Jowz"
},
"source": [
"## Import and parse the training dataset\n",
"\n",
"Download the dataset file and convert it into a structure that can be used by this Python program.\n",
"\n",
"### Download the dataset\n",
"\n",
"Download the training dataset file using the `tf.keras.utils.get_file` function. This returns the file path of the downloaded file:"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4fdWxY0MEiWW",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">Le fichier d'origine est à cette adresse : https://archive.ics.uci.edu/ml/datasets/iris"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LvDQlxnUHEbV",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">[tf.keras.utils.get_file](https://www.tensorflow.org/api_docs/python/tf/keras/utils/get_file) downloads a file from a URL if it's not already in the cache and returns the path to the downloaded file"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "J6c7uEU9rjRM",
"colab": {}
},
"source": [
"train_dataset_url = \"https://storage.googleapis.com/download.tensorflow.org/data/iris_training.csv\"\n",
"train_dataset_fp = tf.keras.utils.get_file(fname=os.path.basename(train_dataset_url),\n",
" origin=train_dataset_url)\n",
"print(\"Local copy of the dataset file: {}\".format(train_dataset_fp))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "bpFEdFTb06lG",
"colab_type": "text"
},
"source": [
"# <font color=\"teal\">sklearn</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "210aYfRkF9VW",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">Etant donné, que ce fichier est un \"classique\" du Machine Learning, on peut le lire avec tous les packages ML, notamment avec sklearn mais la structure retournée ne sera pas la même.\n",
"\n",
"Ce bout de code peut être ignoré."
]
},
{
"cell_type": "code",
"metadata": {
"id": "NXkNu0XxGSck",
"colab_type": "code",
"colab": {}
},
"source": [
"from sklearn.datasets import load_iris\n",
"iris = load_iris()\n",
"print (iris['DESCR'])"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "u9LYVsljIn4s",
"colab_type": "code",
"colab": {}
},
"source": [
"import pandas as pd"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "l8O58XwTJiVn",
"colab_type": "code",
"colab": {}
},
"source": [
"iris_data = pd.DataFrame(data=iris['data'],columns=iris['feature_names'])\n",
"iris_data[\"target\"] = iris['target']"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "O1zypDbeJpxR",
"colab_type": "code",
"colab": {}
},
"source": [
"color_wheel = {\n",
" 1: \"#0392cf\",\n",
" 2: \"#7bc043\",\n",
" 3: \"#ee4035\"}"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "RT_k9egGJyJg",
"colab_type": "code",
"colab": {}
},
"source": [
"colors = iris_data[\"target\"].map(lambda x: color_wheel.get(x + 1))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "xlpo_fgTMRxR",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">A scatter matrix (pairs plot) compactly plots all the numeric variables we have in a dataset against each other one. In Python, this data visualization technique can be carried out with many libraries but if we are using Pandas to load the data, we can use the base scatter_matrix method to visualize the dataset."
]
},
{
"cell_type": "code",
"metadata": {
"id": "crgwlj0IJ2IT",
"colab_type": "code",
"colab": {}
},
"source": [
"ax = pd.plotting.scatter_matrix(iris_data, color=colors, alpha=0.6, figsize=(15, 15))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "qnX1-aLors4S"
},
"source": [
"# Inspect the data\n",
"\n",
"This dataset, `iris_training.csv`, is a plain text file that stores tabular data formatted as comma-separated values (CSV). Use the `head -n5` command to take a peek at the first five entries:"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hDR-9F6lz0FD",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">! permet d'exécuter une commande système"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "FQvb_JYdrpPm",
"colab": {}
},
"source": [
"!head -n5 {train_dataset_fp}"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "kQhzD6P-uBoq"
},
"source": [
"From this view of the dataset, notice the following:\n",
"\n",
"1. The first line is a header containing information about the dataset:\n",
" * There are 120 total examples. Each example has four features and one of three possible label names.\n",
"2. Subsequent rows are data records, one *[example](https://developers.google.com/machine-learning/glossary/#example)* per line, where:\n",
" * The first four fields are *[features](https://developers.google.com/machine-learning/glossary/#feature)*: these are the characteristics of an example. Here, the fields hold float numbers representing flower measurements.\n",
" * The last column is the *[label](https://developers.google.com/machine-learning/glossary/#label)*: this is the value we want to predict. For this dataset, it's an integer value of 0, 1, or 2 that corresponds to a flower name.\n",
"\n",
"Let's write that out in code:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "9Edhevw7exl6",
"colab": {}
},
"source": [
"# column order in CSV file\n",
"column_names = ['sepal_length', 'sepal_width', 'petal_length', 'petal_width', 'species']\n",
"\n",
"feature_names = column_names[:-1]\n",
"label_name = column_names[-1]\n",
"\n",
"print(\"Features: {}\".format(feature_names))\n",
"print(\"Label: {}\".format(label_name))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "CCtwLoJhhDNc"
},
"source": [
"Each label is associated with string name (for example, \"setosa\"), but machine learning typically relies on numeric values. The label numbers are mapped to a named representation, such as:\n",
"\n",
"* `0`: Iris setosa\n",
"* `1`: Iris versicolor\n",
"* `2`: Iris virginica\n",
"\n",
"For more information about features and labels, see the [ML Terminology section of the Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course/framing/ml-terminology)."
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "sVNlJlUOhkoX",
"colab": {}
},
"source": [
"class_names = ['Iris setosa', 'Iris versicolor', 'Iris virginica']"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "dqPkQExM2Pwt"
},
"source": [
"# Create a `tf.data.Dataset`\n",
"\n",
"TensorFlow's [Dataset API](../../guide/data.ipynb) handles many common cases for loading data into a model. This is a high-level API for reading data and transforming it into a form used for training.\n",
"\n",
"\n",
"Since the dataset is a CSV-formatted text file, use the `tf.data.experimental.make_csv_dataset` function to parse the data into a suitable format. Since this function generates data for training models, the default behavior is to shuffle the data (`shuffle=True, shuffle_buffer_size=10000`), and repeat the dataset forever (`num_epochs=None`). We also set the [batch_size](https://developers.google.com/machine-learning/glossary/#batch_size) parameter:"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6w3oOYLYO_sH",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">On lira avec profit l'article [Dataset 5 fleurs](http://tensorflow.backprop.fr/dataset-5-fleurs/) "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mc0WAaCjPywj",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">[tf.data.experimental.make_csv_dataset](https://www.tensorflow.org/api_docs/python/tf/data/experimental/make_csv_dataset) reads CSV files into a dataset and returns a dataset, where each element is a (features, labels) tuple that corresponds to a batch of batch_size CSV rows. \n",
"\n",
"<font color=\"LightSlateGray\">The features dictionary maps feature column names to Tensors containing the corresponding column data, and labels is a Tensor containing the column data for the label column specified by label_name."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2GCcVQ2A8Ud5",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">Par défaut header=True, ce qui permet d'ignorer la 1ère ligne du fichier csv\n",
"\n",
"<font color=\"LightSlateGray\">header: A bool that indicates whether the first rows of provided CSV files correspond to header lines with column names, and should not be included in the data"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EUaPH1hEGm8E",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">On crée le dataset en lisant le fichier csv, en ignorant la 1ère ligne. On a 5 colonnes et on précise que la colonne label est la dernière."
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "WsxHnz1ebJ2S",
"colab": {}
},
"source": [
"batch_size = 32\n",
"\n",
"train_dataset = tf.data.experimental.make_csv_dataset(\n",
" train_dataset_fp,\n",
" batch_size,\n",
" column_names=column_names,\n",
" label_name=label_name,\n",
" num_epochs=1)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "gB_RSn62c-3G"
},
"source": [
"The `make_csv_dataset` function returns a `tf.data.Dataset` of `(features, label)` pairs, where `features` is a dictionary: `{'feature_name': value}`\n",
"\n",
"These `Dataset` objects are iterable. Let's look at a batch of features:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "iDuG94H-C122",
"colab": {}
},
"source": [
"features, labels = next(iter(train_dataset))\n",
"print(features)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "maJD3cgwbpeJ",
"colab_type": "code",
"colab": {}
},
"source": [
"print(labels)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "MNfoNwCLHsrn",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">Une autre façon de voir un \"batch of features\""
]
},
{
"cell_type": "code",
"metadata": {
"id": "kOAEOzOZ9I1J",
"colab_type": "code",
"colab": {}
},
"source": [
"takeone = train_dataset.take(1)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "VtPpEO9e-ZGa",
"colab_type": "code",
"colab": {}
},
"source": [
"for data in takeone:\n",
" print(data)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "LwEX6d_0AyMn",
"colab_type": "code",
"colab": {}
},
"source": [
"data[0]['sepal_length']"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "E63mArnQaAGz"
},
"source": [
"Notice that like-features are grouped together, or *batched*. Each example row's fields are appended to the corresponding feature array. Change the `batch_size` to set the number of examples stored in these feature arrays.\n",
"\n",
"You can start to see some clusters by plotting a few features from the batch:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "me5Wn-9FcyyO",
"colab": {}
},
"source": [
"plt.scatter(features['petal_length'],\n",
" features['sepal_length'],\n",
" c=labels,\n",
" cmap='viridis')\n",
"\n",
"plt.xlabel(\"Petal length\")\n",
"plt.ylabel(\"Sepal length\")\n",
"plt.show()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "YlxpSyHlhT6M"
},
"source": [
"To simplify the model building step, create a function to repackage the features dictionary into a single array with shape: `(batch_size, num_features)`.\n",
"\n",
"This function uses the `tf.stack` method which takes values from a list of tensors and creates a combined tensor at the specified dimension:"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CzSJZfASTfPf",
"colab_type": "text"
},
"source": [
"## <font color=\"orange\">tf.stack</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "naoAfPuGIvAV",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">tf.stack stacks a list of rank-R tensors into one rank-(R+1) tensor.\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "KgQwD233KRW7",
"colab_type": "code",
"colab": {}
},
"source": [
"x = tf.constant([1, 4]) \n",
"y = tf.constant([2, 5]) \n",
"z = tf.constant([3, 6]) \n",
"tf.stack([x, y, z]) "
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "DIFB3Al9LoXY",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)\n",
"\n",
"<font color=\"LightSlateGray\">Given a list of length N of tensors of shape (A, B, C);\n",
"\n",
"<font color=\"LightSlateGray\">if axis == 0 then the output tensor will have the shape (N, A, B, C). if axis == 1 then the output tensor will have the shape (A, N, B, C)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "GTHAYjoKKonu",
"colab_type": "code",
"colab": {}
},
"source": [
"tf.stack([x, y, z], axis=1)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "RrktuhMfT52D",
"colab_type": "text"
},
"source": [
"## <font color=\"orange\">pack_features_vector</font>"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "jm932WINcaGU",
"colab": {}
},
"source": [
"def pack_features_vector(features, labels):\n",
" \"\"\"Pack the features into a single array.\"\"\"\n",
" features = tf.stack(list(features.values()), axis=1)\n",
" return features, labels"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "V1Vuph_eDl8x"
},
"source": [
"Then use the `tf.data.Dataset#map` method to pack the `features` of each `(features,label)` pair into the training dataset:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "ZbDkzGZIkpXf",
"colab": {}
},
"source": [
"train_dataset = train_dataset.map(pack_features_vector)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "NLy0Q1xCldVO"
},
"source": [
"The features element of the `Dataset` are now arrays with shape `(batch_size, num_features)`. Let's look at the first few examples:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "kex9ibEek6Tr",
"colab": {}
},
"source": [
"features, labels = next(iter(train_dataset))\n",
"print(features[:5])"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "5SKHXgg9MK2Y",
"colab_type": "code",
"colab": {}
},
"source": [
"print (labels)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "PadGVBaXUBaU",
"colab_type": "text"
},
"source": [
"## <font color=\"orange\">le dataset réorganisé</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mpcfNmnNNBe_",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "KGSkA7pRM0Pl",
"colab_type": "code",
"colab": {}
},
"source": [
"takeone = train_dataset.take(1)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "dbDOwIWnM4Zq",
"colab_type": "code",
"colab": {}
},
"source": [
"for data in takeone:\n",
" print(data)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "3sa0OPLHNN59",
"colab_type": "code",
"colab": {}
},
"source": [
"data[0][0]"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "j2UbdWZoNWnb",
"colab_type": "code",
"colab": {}
},
"source": [
"data[1][0]"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "LsaVrtNM3Tx5"
},
"source": [
"# Select the type of model\n",
"\n",
"## Why model?\n",
"\n",
"A *[model](https://developers.google.com/machine-learning/crash-course/glossary#model)* is a relationship between features and the label. For the Iris classification problem, the model defines the relationship between the sepal and petal measurements and the predicted Iris species. Some simple models can be described with a few lines of algebra, but complex machine learning models have a large number of parameters that are difficult to summarize.\n",
"\n",
"Could you determine the relationship between the four features and the Iris species *without* using machine learning? That is, could you use traditional programming techniques (for example, a lot of conditional statements) to create a model? Perhaps—if you analyzed the dataset long enough to determine the relationships between petal and sepal measurements to a particular species. And this becomes difficult—maybe impossible—on more complicated datasets. A good machine learning approach *determines the model for you*. If you feed enough representative examples into the right machine learning model type, the program will figure out the relationships for you.\n",
"\n",
"### Select the model\n",
"\n",
"We need to select the kind of model to train. There are many types of models and picking a good one takes experience. This tutorial uses a neural network to solve the Iris classification problem. *[Neural networks](https://developers.google.com/machine-learning/glossary/#neural_network)* can find complex relationships between features and the label. It is a highly-structured graph, organized into one or more *[hidden layers](https://developers.google.com/machine-learning/glossary/#hidden_layer)*. Each hidden layer consists of one or more *[neurons](https://developers.google.com/machine-learning/glossary/#neuron)*. There are several categories of neural networks and this program uses a dense, or *[fully-connected neural network](https://developers.google.com/machine-learning/glossary/#fully_connected_layer)*: the neurons in one layer receive input connections from *every* neuron in the previous layer. For example, Figure 2 illustrates a dense neural network consisting of an input layer, two hidden layers, and an output layer:\n",
"\n",
"<table>\n",
" <tr><td>\n",
" <img src=\"https://www.tensorflow.org/images/custom_estimators/full_network.png\"\n",
" alt=\"A diagram of the network architecture: Inputs, 2 hidden layers, and outputs\">\n",
" </td></tr>\n",
" <tr><td align=\"center\">\n",
" <b>Figure 2.</b> A neural network with features, hidden layers, and predictions.<br/>&nbsp;\n",
" </td></tr>\n",
"</table>\n",
"\n",
"When the model from Figure 2 is trained and fed an unlabeled example, it yields three predictions: the likelihood that this flower is the given Iris species. This prediction is called *[inference](https://developers.google.com/machine-learning/crash-course/glossary#inference)*. For this example, the sum of the output predictions is 1.0. In Figure 2, this prediction breaks down as: `0.02` for *Iris setosa*, `0.95` for *Iris versicolor*, and `0.03` for *Iris virginica*. This means that the model predicts—with 95% probability—that an unlabeled example flower is an *Iris versicolor*."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "W23DIMVPQEBt"
},
"source": [
"### Create a model using Keras\n",
"\n",
"The TensorFlow `tf.keras` API is the preferred way to create models and layers. This makes it easy to build models and experiment while Keras handles the complexity of connecting everything together.\n",
"\n",
"The `tf.keras.Sequential` model is a linear stack of layers. Its constructor takes a list of layer instances, in this case, two `tf.keras.layers.Dense` layers with 10 nodes each, and an output layer with 3 nodes representing our label predictions. The first layer's `input_shape` parameter corresponds to the number of features from the dataset, and is required:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "2fZ6oL2ig3ZK",
"colab": {}
},
"source": [
"model = tf.keras.Sequential([\n",
" tf.keras.layers.Dense(10, activation=tf.nn.relu, input_shape=(4,)), # input shape required\n",
" tf.keras.layers.Dense(10, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(3)\n",
"])"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "FHcbEzMpxbHL"
},
"source": [
"The *[activation function](https://developers.google.com/machine-learning/crash-course/glossary#activation_function)* determines the output shape of each node in the layer. These non-linearities are important—without them the model would be equivalent to a single layer. There are many `tf.keras.activations`, but [ReLU](https://developers.google.com/machine-learning/crash-course/glossary#ReLU) is common for hidden layers.\n",
"\n",
"The ideal number of hidden layers and neurons depends on the problem and the dataset. Like many aspects of machine learning, picking the best shape of the neural network requires a mixture of knowledge and experimentation. As a rule of thumb, increasing the number of hidden layers and neurons typically creates a more powerful model, which requires more data to train effectively."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "MWzfg8vfXMxk",
"colab_type": "text"
},
"source": [
"### <font color=\"green\">façon de faire habituelle</font>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sn0S7FaOYwDI",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "MRR_L-OyHC-k",
"colab_type": "code",
"colab": {}
},
"source": [
"model_2 = tf.keras.Sequential([\n",
" tf.keras.layers.Dense(10, activation=tf.nn.relu, input_shape=(4,)), # input shape required\n",
" tf.keras.layers.Dense(10, activation=tf.nn.relu),\n",
" tf.keras.layers.Dense(3, activation=tf.nn.softmax)\n",
"])"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "hcASI3YCYRer",
"colab_type": "code",
"colab": {}
},
"source": [
"model_2.compile(optimizer='SGD',\n",
" loss='sparse_categorical_crossentropy',\n",
" metrics=['accuracy'])"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "VIcVmbrCXles",
"colab_type": "code",
"colab": {}
},
"source": [
"model_2.summary()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "Qz8tgeK5NTzN",
"colab_type": "code",
"colab": {}
},
"source": [
"%%time\n",
"history = model_2.fit(train_dataset, epochs=100)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "2wFKnhWCpDSS"
},
"source": [
"### Using the model\n",
"\n",
"Let's have a quick look at what this model does to a batch of features:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "xe6SQ5NrpB-I",
"colab": {}
},
"source": [
"predictions = model_2(features)\n",
"predictions[:5]"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "micUCJG7ZouC",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "aSopEiO0ZwlO",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">logit, softmax et argmax permettent d'expliquer ce qui est fait par le réseau"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dDajsH7BdslK",
"colab_type": "text"
},
"source": [
"<font color=\"LightSlateGray\">La dernière couche du réseau de neurones, est la couche logit. C'est la sortie du réseau. Si comme ici on a 3 classes alors on aura 3 neurones, 3 valeurs en output. On applique ensuite un softmax pour avoir des probabilités puis on prend la valeur la plus grande pour avoir notre classe."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "wxyXOhwVr5S3"
},
"source": [
"Here, each example returns a [logit](https://developers.google.com/machine-learning/crash-course/glossary#logits) for each class.\n",
"\n",
"To convert these logits to a probability for each class, use the [softmax](https://developers.google.com/machine-learning/crash-course/glossary#softmax) function:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "_tRwHZmTNTX2",
"colab": {}
},
"source": [
"tf.nn.softmax(predictions[:5])"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "uRZmchElo481"
},
"source": [
"Taking the `tf.argmax` across classes gives us the predicted class index. But, the model hasn't been trained yet, so these aren't good predictions:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "-Jzm_GoErz8B",
"colab": {}
},
"source": [
"print(\"Prediction: {}\".format(tf.argmax(predictions, axis=1)))\n",
"print(\" Labels: {}\".format(labels))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Vzq2E5J2QMtw"
},
"source": [
"## Train the model\n",
"\n",
"*[Training](https://developers.google.com/machine-learning/crash-course/glossary#training)* is the stage of machine learning when the model is gradually optimized, or the model *learns* the dataset. The goal is to learn enough about the structure of the training dataset to make predictions about unseen data. If you learn *too much* about the training dataset, then the predictions only work for the data it has seen and will not be generalizable. This problem is called *[overfitting](https://developers.google.com/machine-learning/crash-course/glossary#overfitting)*—it's like memorizing the answers instead of understanding how to solve a problem.\n",
"\n",
"The Iris classification problem is an example of *[supervised machine learning](https://developers.google.com/machine-learning/glossary/#supervised_machine_learning)*: the model is trained from examples that contain labels. In *[unsupervised machine learning](https://developers.google.com/machine-learning/glossary/#unsupervised_machine_learning)*, the examples don't contain labels. Instead, the model typically finds patterns among the features."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "RaKp8aEjKX6B"
},
"source": [
"### Define the loss and gradient function\n",
"\n",
"Both training and evaluation stages need to calculate the model's *[loss](https://developers.google.com/machine-learning/crash-course/glossary#loss)*. This measures how off a model's predictions are from the desired label, in other words, how bad the model is performing. We want to minimize, or optimize, this value.\n",
"\n",
"Our model will calculate its loss using the `tf.keras.losses.SparseCategoricalCrossentropy` function which takes the model's class probability predictions and the desired label, and returns the average loss across the examples."
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "QOsi6b-1CXIn",
"colab": {}
},
"source": [
"loss_object = tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "tMAT4DcMPwI-",
"colab": {}
},
"source": [
"def loss(model, x, y, training):\n",
" # training=training is needed only if there are layers with different\n",
" # behavior during training versus inference (e.g. Dropout).\n",
" y_ = model(x, training=training)\n",
"\n",
" return loss_object(y_true=y, y_pred=y_)\n",
"\n",
"\n",
"l = loss(model, features, labels, training=False)\n",
"print(\"Loss test: {}\".format(l))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "3IcPqA24QM6B"
},
"source": [
"Use the `tf.GradientTape` context to calculate the *[gradients](https://developers.google.com/machine-learning/crash-course/glossary#gradient)* used to optimize your model:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "x57HcKWhKkei",
"colab": {}
},
"source": [
"def grad(model, inputs, targets):\n",
" with tf.GradientTape() as tape:\n",
" loss_value = loss(model, inputs, targets, training=True)\n",
" return loss_value, tape.gradient(loss_value, model.trainable_variables)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "lOxFimtlKruu"
},
"source": [
"### Create an optimizer\n",
"\n",
"An *[optimizer](https://developers.google.com/machine-learning/crash-course/glossary#optimizer)* applies the computed gradients to the model's variables to minimize the `loss` function. You can think of the loss function as a curved surface (see Figure 3) and we want to find its lowest point by walking around. The gradients point in the direction of steepest ascent—so we'll travel the opposite way and move down the hill. By iteratively calculating the loss and gradient for each batch, we'll adjust the model during training. Gradually, the model will find the best combination of weights and bias to minimize loss. And the lower the loss, the better the model's predictions.\n",
"\n",
"<table>\n",
" <tr><td>\n",
" <img src=\"https://cs231n.github.io/assets/nn3/opt1.gif\" width=\"70%\"\n",
" alt=\"Optimization algorithms visualized over time in 3D space.\">\n",
" </td></tr>\n",
" <tr><td align=\"center\">\n",
" <b>Figure 3.</b> Optimization algorithms visualized over time in 3D space.<br/>(Source: <a href=\"http://cs231n.github.io/neural-networks-3/\">Stanford class CS231n</a>, MIT License, Image credit: <a href=\"https://twitter.com/alecrad\">Alec Radford</a>)\n",
" </td></tr>\n",
"</table>\n",
"\n",
"TensorFlow has many optimization algorithms available for training. This model uses the `tf.keras.optimizers.SGD` that implements the *[stochastic gradient descent](https://developers.google.com/machine-learning/crash-course/glossary#gradient_descent)* (SGD) algorithm. The `learning_rate` sets the step size to take for each iteration down the hill. This is a *hyperparameter* that you'll commonly adjust to achieve better results."
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "XkUd6UiZa_dF"
},
"source": [
"Let's setup the optimizer:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "8xxi2NNGKwG_",
"colab": {}
},
"source": [
"optimizer = tf.keras.optimizers.SGD(learning_rate=0.01)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "pJVRZ0hP52ZB"
},
"source": [
"We'll use this to calculate a single optimization step:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "rxRNTFVe56RG",
"colab": {}
},
"source": [
"loss_value, grads = grad(model, features, labels)\n",
"\n",
"print(\"Step: {}, Initial Loss: {}\".format(optimizer.iterations.numpy(),\n",
" loss_value.numpy()))\n",
"\n",
"optimizer.apply_gradients(zip(grads, model.trainable_variables))\n",
"\n",
"print(\"Step: {}, Loss: {}\".format(optimizer.iterations.numpy(),\n",
" loss(model, features, labels, training=True).numpy()))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "7Y2VSELvwAvW"
},
"source": [
"### Training loop\n",
"\n",
"With all the pieces in place, the model is ready for training! A training loop feeds the dataset examples into the model to help it make better predictions. The following code block sets up these training steps:\n",
"\n",
"1. Iterate each *epoch*. An epoch is one pass through the dataset.\n",
"2. Within an epoch, iterate over each example in the training `Dataset` grabbing its *features* (`x`) and *label* (`y`).\n",
"3. Using the example's features, make a prediction and compare it with the label. Measure the inaccuracy of the prediction and use that to calculate the model's loss and gradients.\n",
"4. Use an `optimizer` to update the model's variables.\n",
"5. Keep track of some stats for visualization.\n",
"6. Repeat for each epoch.\n",
"\n",
"The `num_epochs` variable is the number of times to loop over the dataset collection. Counter-intuitively, training a model longer does not guarantee a better model. `num_epochs` is a *[hyperparameter](https://developers.google.com/machine-learning/glossary/#hyperparameter)* that you can tune. Choosing the right number usually requires both experience and experimentation:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "AIgulGRUhpto",
"colab": {}
},
"source": [
"## Note: Rerunning this cell uses the same model variables\n",
"\n",
"# Keep results for plotting\n",
"train_loss_results = []\n",
"train_accuracy_results = []\n",
"\n",
"num_epochs = 201\n",
"\n",
"for epoch in range(num_epochs):\n",
" epoch_loss_avg = tf.keras.metrics.Mean()\n",
" epoch_accuracy = tf.keras.metrics.SparseCategoricalAccuracy()\n",
"\n",
" # Training loop - using batches of 32\n",
" for x, y in train_dataset:\n",
" # Optimize the model\n",
" loss_value, grads = grad(model, x, y)\n",
" optimizer.apply_gradients(zip(grads, model.trainable_variables))\n",
"\n",
" # Track progress\n",
" epoch_loss_avg.update_state(loss_value) # Add current batch loss\n",
" # Compare predicted label to actual label\n",
" # training=True is needed only if there are layers with different\n",
" # behavior during training versus inference (e.g. Dropout).\n",
" epoch_accuracy.update_state(y, model(x, training=True))\n",
"\n",
" # End epoch\n",
" train_loss_results.append(epoch_loss_avg.result())\n",
" train_accuracy_results.append(epoch_accuracy.result())\n",
"\n",
" if epoch % 50 == 0:\n",
" print(\"Epoch {:03d}: Loss: {:.3f}, Accuracy: {:.3%}\".format(epoch,\n",
" epoch_loss_avg.result(),\n",
" epoch_accuracy.result()))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "SGnCBctFXRF3",
"colab_type": "text"
},
"source": [
"[![](https://raw.githubusercontent.com/BackProp-fr/meetup/master/images/LogoBackPropTranspSmall.png)](https://www.backprop.fr)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_jRzm-HyW_s7",
"colab_type": "text"
},
"source": [
"### <font color=\"green\">Avec TensorBoard</font>"
]
},
{
"cell_type": "code",
"metadata": {
"id": "QFVfCziGYOIq",
"colab_type": "code",
"colab": {}
},
"source": [
"# Load the TensorBoard notebook extension\n",
"%load_ext tensorboard"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "v3V_i7NTYTeJ",
"colab_type": "code",
"colab": {}
},
"source": [
"# Clear any logs from previous runs\n",
"!rm -rf ./logs/ "
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "8Zn7E3OSZD1O",
"colab_type": "text"
},
"source": [
"Set up summary writers to write the summaries to disk in a different logs directory:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "FV0TVsr9ZCPX",
"colab_type": "code",
"colab": {}
},
"source": [
"import datetime\n",
"current_time = datetime.datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n",
"train_log_dir = 'logs/gradient_tape/' + current_time + '/train'\n",
"test_log_dir = 'logs/gradient_tape/' + current_time + '/test'\n",
"train_summary_writer = tf.summary.create_file_writer(train_log_dir)\n",
"test_summary_writer = tf.summary.create_file_writer(test_log_dir)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "cDbe-vLubUU-",
"colab_type": "code",
"colab": {}
},
"source": [
"# Define our metrics\n",
"train_loss = tf.keras.metrics.Mean('train_loss', dtype=tf.float32)\n",
"train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy('train_accuracy')\n",
"test_loss = tf.keras.metrics.Mean('test_loss', dtype=tf.float32)\n",
"test_accuracy = tf.keras.metrics.SparseCategoricalAccuracy('test_accuracy')"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "usfd2KKucnaE",
"colab_type": "code",
"colab": {}
},
"source": [
"## Note: Rerunning this cell uses the same model variables\n",
"\n",
"# Keep results for plotting\n",
"train_loss_results = []\n",
"train_accuracy_results = []\n",
"\n",
"num_epochs = 201\n",
"\n",
"for epoch in range(num_epochs):\n",
"\n",
" # Training loop - using batches of 32\n",
" for x, y in train_dataset:\n",
" # Optimize the model\n",
" loss_value, grads = grad(model, x, y)\n",
" optimizer.apply_gradients(zip(grads, model.trainable_variables))\n",
"\n",
" # Track progress\n",
" train_loss.update_state(loss_value) # Add current batch loss\n",
" # Compare predicted label to actual label\n",
" # training=True is needed only if there are layers with different\n",
" # behavior during training versus inference (e.g. Dropout).\n",
" train_accuracy.update_state(y, model(x, training=True))\n",
"\n",
" # End epoch\n",
" train_loss_results.append(train_loss.result())\n",
" train_accuracy_results.append(train_accuracy.result())\n",
"\n",
" with train_summary_writer.as_default():\n",
" tf.summary.scalar('loss', train_loss.result(), step=epoch)\n",
" tf.summary.scalar('accuracy', train_accuracy.result(), step=epoch)\n",
"\n",
" if epoch % 50 == 0:\n",
" print(\"Epoch {:03d}: Loss: {:.3f}, Accuracy: {:.3%}\".format(epoch,\n",
" train_loss.result(),\n",
" train_accuracy.result()))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "k2-6d4xygf1E",
"colab_type": "code",
"colab": {}
},
"source": [
"%tensorboard --logdir logs/gradient_tape"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "SdTbR5_dbuua",
"colab_type": "text"
},
"source": [
"Define the training and test functions:"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "2FQHVUnm_rjw"
},
"source": [
"### Visualize the loss function over time"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "j3wdbmtLVTyr"
},
"source": [
"While it's helpful to print out the model's training progress, it's often *more* helpful to see this progress. [TensorBoard](https://www.tensorflow.org/tensorboard) is a nice visualization tool that is packaged with TensorFlow, but we can create basic charts using the `matplotlib` module.\n",
"\n",
"Interpreting these charts takes some experience, but you really want to see the *loss* go down and the *accuracy* go up:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "agjvNd2iUGFn",
"colab": {}
},
"source": [
"fig, axes = plt.subplots(2, sharex=True, figsize=(12, 8))\n",
"fig.suptitle('Training Metrics')\n",
"\n",
"axes[0].set_ylabel(\"Loss\", fontsize=14)\n",
"axes[0].plot(train_loss_results)\n",
"\n",
"axes[1].set_ylabel(\"Accuracy\", fontsize=14)\n",
"axes[1].set_xlabel(\"Epoch\", fontsize=14)\n",
"axes[1].plot(train_accuracy_results)\n",
"plt.show()"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "Zg8GoMZhLpGH"
},
"source": [
"## Evaluate the model's effectiveness\n",
"\n",
"Now that the model is trained, we can get some statistics on its performance.\n",
"\n",
"*Evaluating* means determining how effectively the model makes predictions. To determine the model's effectiveness at Iris classification, pass some sepal and petal measurements to the model and ask the model to predict what Iris species they represent. Then compare the model's predictions against the actual label. For example, a model that picked the correct species on half the input examples has an *[accuracy](https://developers.google.com/machine-learning/glossary/#accuracy)* of `0.5`. Figure 4 shows a slightly more effective model, getting 4 out of 5 predictions correct at 80% accuracy:\n",
"\n",
"<table cellpadding=\"8\" border=\"0\">\n",
" <colgroup>\n",
" <col span=\"4\" >\n",
" <col span=\"1\" bgcolor=\"lightblue\">\n",
" <col span=\"1\" bgcolor=\"lightgreen\">\n",
" </colgroup>\n",
" <tr bgcolor=\"lightgray\">\n",
" <th colspan=\"4\">Example features</th>\n",
" <th colspan=\"1\">Label</th>\n",
" <th colspan=\"1\" >Model prediction</th>\n",
" </tr>\n",
" <tr>\n",
" <td>5.9</td><td>3.0</td><td>4.3</td><td>1.5</td><td align=\"center\">1</td><td align=\"center\">1</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6.9</td><td>3.1</td><td>5.4</td><td>2.1</td><td align=\"center\">2</td><td align=\"center\">2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>5.1</td><td>3.3</td><td>1.7</td><td>0.5</td><td align=\"center\">0</td><td align=\"center\">0</td>\n",
" </tr>\n",
" <tr>\n",
" <td>6.0</td> <td>3.4</td> <td>4.5</td> <td>1.6</td> <td align=\"center\">1</td><td align=\"center\" bgcolor=\"red\">2</td>\n",
" </tr>\n",
" <tr>\n",
" <td>5.5</td><td>2.5</td><td>4.0</td><td>1.3</td><td align=\"center\">1</td><td align=\"center\">1</td>\n",
" </tr>\n",
" <tr><td align=\"center\" colspan=\"6\">\n",
" <b>Figure 4.</b> An Iris classifier that is 80% accurate.<br/>&nbsp;\n",
" </td></tr>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "z-EvK7hGL0d8"
},
"source": [
"### Setup the test dataset\n",
"\n",
"Evaluating the model is similar to training the model. The biggest difference is the examples come from a separate *[test set](https://developers.google.com/machine-learning/crash-course/glossary#test_set)* rather than the training set. To fairly assess a model's effectiveness, the examples used to evaluate a model must be different from the examples used to train the model.\n",
"\n",
"The setup for the test `Dataset` is similar to the setup for training `Dataset`. Download the CSV text file and parse that values, then give it a little shuffle:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "Ps3_9dJ3Lodk",
"colab": {}
},
"source": [
"test_url = \"https://storage.googleapis.com/download.tensorflow.org/data/iris_test.csv\"\n",
"test_fp = tf.keras.utils.get_file(fname=os.path.basename(test_url),\n",
" origin=test_url)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "SRMWCu30bnxH",
"colab": {}
},
"source": [
"test_dataset = tf.data.experimental.make_csv_dataset(\n",
" test_fp,\n",
" batch_size,\n",
" column_names=column_names,\n",
" label_name='species',\n",
" num_epochs=1,\n",
" shuffle=False)\n",
"\n",
"test_dataset = test_dataset.map(pack_features_vector)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "HFuOKXJdMAdm"
},
"source": [
"### Evaluate the model on the test dataset\n",
"\n",
"Unlike the training stage, the model only evaluates a single [epoch](https://developers.google.com/machine-learning/glossary/#epoch) of the test data. In the following code cell, we iterate over each example in the test set and compare the model's prediction against the actual label. This is used to measure the model's accuracy across the entire test set:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "Tw03-MK1cYId",
"colab": {}
},
"source": [
"test_accuracy = tf.keras.metrics.Accuracy()\n",
"\n",
"for (x, y) in test_dataset:\n",
" # training=False is needed only if there are layers with different\n",
" # behavior during training versus inference (e.g. Dropout).\n",
" logits = model(x, training=False)\n",
" prediction = tf.argmax(logits, axis=1, output_type=tf.int32)\n",
" test_accuracy(prediction, y)\n",
"\n",
"print(\"Test set accuracy: {:.3%}\".format(test_accuracy.result()))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "HcKEZMtCOeK-"
},
"source": [
"We can see on the last batch, for example, the model is usually correct:"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "uNwt2eMeOane",
"colab": {}
},
"source": [
"tf.stack([y,prediction],axis=1)"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"colab_type": "text",
"id": "7Li2r1tYvW7S"
},
"source": [
"## Use the trained model to make predictions\n",
"\n",
"We've trained a model and \"proven\" that it's good—but not perfect—at classifying Iris species. Now let's use the trained model to make some predictions on [unlabeled examples](https://developers.google.com/machine-learning/glossary/#unlabeled_example); that is, on examples that contain features but not a label.\n",
"\n",
"In real-life, the unlabeled examples could come from lots of different sources including apps, CSV files, and data feeds. For now, we're going to manually provide three unlabeled examples to predict their labels. Recall, the label numbers are mapped to a named representation as:\n",
"\n",
"* `0`: Iris setosa\n",
"* `1`: Iris versicolor\n",
"* `2`: Iris virginica"
]
},
{
"cell_type": "code",
"metadata": {
"colab_type": "code",
"id": "kesTS5Lzv-M2",
"colab": {}
},
"source": [
"predict_dataset = tf.convert_to_tensor([\n",
" [5.1, 3.3, 1.7, 0.5,],\n",
" [5.9, 3.0, 4.2, 1.5,],\n",
" [6.9, 3.1, 5.4, 2.1]\n",
"])\n",
"\n",
"# training=False is needed only if there are layers with different\n",
"# behavior during training versus inference (e.g. Dropout).\n",
"predictions = model(predict_dataset, training=False)\n",
"\n",
"for i, logits in enumerate(predictions):\n",
" class_idx = tf.argmax(logits).numpy()\n",
" p = tf.nn.softmax(logits)[class_idx]\n",
" name = class_names[class_idx]\n",
" print(\"Example {} prediction: {} ({:4.1f}%)\".format(i, name, 100*p))"
],
"execution_count": 0,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "TQLA40CD-SgQ",
"colab_type": "code",
"colab": {}
},
"source": [
""
],
"execution_count": 0,
"outputs": []
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment