Skip to content

Instantly share code, notes, and snippets.

@PaulSayantan

PaulSayantan/tensorflow_basics.ipynb

Last active August 4, 2020 17:21
Show Gist options
  • Save PaulSayantan/779fb44c9f533062e03868c8b01e859f to your computer and use it in GitHub Desktop.
Save PaulSayantan/779fb44c9f533062e03868c8b01e859f to your computer and use it in GitHub Desktop.
Download ZIP
Tensorflow
Display the source blob
Display the rendered blob
Raw
tensorflow_basics.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "tensorflow-basics.ipynb",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "IwkK3jqVNZWE",
"colab_type": "text"
},
"source": [
"# <h1 align=\"center\">Tensorflow For Beginners</h1>\n",
"\n",
"<h6 align=\"right\">𝓢𝓪𝔂𝓪𝓷𝓽𝓪𝓷 𝓟𝓪𝓾𝓵</h6>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "97JiPuTIRnue",
"colab_type": "text"
},
"source": [
"## Install Tensorflow"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KrtAtRtPRghC",
"colab_type": "text"
},
"source": [
"To install tensorflow in local machine\n",
"\n",
"~~~\n",
"pip install -q tensorflow tensorflow-addons tensorflow-gpu\n",
"~~~"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "AHPTAvG_T5v7",
"colab_type": "text"
},
"source": [
"## Import Tensorflow"
]
},
{
"cell_type": "code",
"metadata": {
"id": "5TNjDOQ0TQvu",
"colab_type": "code",
"colab": {}
},
"source": [
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import tensorflow as tf\n",
"import cProfile"
],
"execution_count": 1,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "JX8ec78tEkwx",
"colab_type": "text"
},
"source": [
"# <h1 align=\"center\">-----Part - 1-----</h1>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xe4A9AtQEmLJ",
"colab_type": "text"
},
"source": [
"## Eager execution\n",
"\n",
"TensorFlow's eager execution is an imperative programming environment that evaluates operations immediately, without building graphs: operations return concrete values instead of constructing a computational graph to run later. This makes it easy to get started with TensorFlow and debug models, and it reduces boilerplate as well. To follow along with this guide, run the code samples below in an interactive python interpreter.\n",
"\n",
"Eager execution is a flexible machine learning platform for research and experimentation, providing:\n",
"\n",
" - An **intuitive interface** — Structure your code naturally and use Python data structures. Quickly iterate on small models and small data.\n",
" - **Easier debugging** — Call ops directly to inspect running models and test changes. Use standard Python debugging tools for immediate error reporting.\n",
" - **Natural control flow** — Use Python control flow instead of graph control flow, simplifying the specification of dynamic models.\n",
"\n",
"Eager execution supports most TensorFlow operations and GPU acceleration."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sVADt9EWFH70",
"colab_type": "text"
},
"source": [
"In Tensorflow 2.0, eager execution is enabled by default."
]
},
{
"cell_type": "code",
"metadata": {
"id": "XFME4pXGFQaH",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "17bcf5d3-f7ec-4294-dbbb-e0aff69642b6"
},
"source": [
"tf.executing_eagerly()"
],
"execution_count": 2,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"True"
]
},
"metadata": {
"tags": []
},
"execution_count": 2
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ISciwl9fFYPv",
"colab_type": "text"
},
"source": [
"Now you can run TensorFlow operations and the results will return immediately:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "p68paqllFZud",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "f8418c95-dcd0-4a84-f835-d0435dc40ddd"
},
"source": [
"x = [[2.]]\n",
"m = tf.matmul(x, x)\n",
"print(\"hello, {}\".format(m))"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"hello, [[4.]]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CSv0MgWBFg1P",
"colab_type": "text"
},
"source": [
"Eager execution works nicely with NumPy. NumPy operations accept `tf.Tensor` arguments. The TensorFlow `tf.math` operations convert Python objects and NumPy arrays to `tf.Tensor` objects. The `tf.Tensor.numpy` method returns the object's value as a NumPy `ndarray`."
]
},
{
"cell_type": "code",
"metadata": {
"id": "Z5EpgU-rFupc",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "15f78ef0-7dcc-4f7e-a449-4c13bef0d921"
},
"source": [
"a = tf.constant([[1, 2],\n",
" [3, 4]])\n",
"print(a)"
],
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[1 2]\n",
" [3 4]], shape=(2, 2), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "563JWqChF03W",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "b5ba034a-f015-4ad7-dafa-b8ac73764c5e"
},
"source": [
"# Broadcasting support\n",
"b = tf.add(a, 1)\n",
"print(b)"
],
"execution_count": 5,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[2 3]\n",
" [4 5]], shape=(2, 2), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "1qI3F6dwF5PV",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "5e79f471-e802-46c1-df76-3746c7396ed8"
},
"source": [
"# Operator overloading is supported\n",
"print(a * b)"
],
"execution_count": 6,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[ 2 6]\n",
" [12 20]], shape=(2, 2), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "y20Ic3HpUxBy",
"colab_type": "text"
},
"source": [
"## Create Tensors\n",
"\n",
"**What is a Tensor?**\n",
"Tensors are multi-dimensional arrays with a uniform type (called a `dtype`). You can see all supported dtypes at `tf.dtypes.DType`.\n",
"\n",
"If you're familiar with NumPy, tensors are (kind of) like `np.arrays`.\n",
"\n",
"All tensors are immutable like python numbers and strings: you can never update the contents of a tensor, only create a new one."
]
},
{
"cell_type": "code",
"metadata": {
"id": "j3DNdSTDUwXX",
"colab_type": "code",
"colab": {}
},
"source": [
"# a string tensor variable\n",
"string = tf.Variable(\"Tensorflow is awesome!\", tf.string)\n",
"\n",
"# a integer tensor variable\n",
"num = tf.Variable(324, tf.int32)\n",
"\n",
"# a floating tensor variable\n",
"decimal = tf.Variable(3.256, tf.float64)\n",
"\n",
"# a boolean tensor variable\n",
"boolean = tf.Variable([False, False, False, True])\n",
"\n",
"# a complex tensor variable\n",
"complex_var = tf.Variable([5 + 4j, 6 + 1j])"
],
"execution_count": 7,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "fcE_YGXRVUcF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 100
},
"outputId": "c5b08ace-d1e4-489a-836c-898a6cc5f77f"
},
"source": [
"print(string)\n",
"\n",
"print(num)\n",
"\n",
"print(decimal)\n",
"\n",
"print(boolean)\n",
"\n",
"print(complex_var)"
],
"execution_count": 8,
"outputs": [
{
"output_type": "stream",
"text": [
"<tf.Variable 'Variable:0' shape=() dtype=string, numpy=b'Tensorflow is awesome!'>\n",
"<tf.Variable 'Variable:0' shape=() dtype=int32, numpy=324>\n",
"<tf.Variable 'Variable:0' shape=() dtype=float32, numpy=3.256>\n",
"<tf.Variable 'Variable:0' shape=(4,) dtype=bool, numpy=array([False, False, False, True])>\n",
"<tf.Variable 'Variable:0' shape=(2,) dtype=complex128, numpy=array([5.+4.j, 6.+1.j])>\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GBSalmJfaD34",
"colab_type": "text"
},
"source": [
"Most tensor operations work on variables as expected, although variables cannot be reshaped."
]
},
{
"cell_type": "code",
"metadata": {
"id": "JXKfWk46aF6g",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 183
},
"outputId": "414f7f60-dc59-4ccc-b21d-de01e771e875"
},
"source": [
"my_tensor = tf.constant([[1.0, 2.0], [3.0, 4.0]])\n",
"my_var = tf.Variable(my_tensor)\n",
"\n",
"print(\"\\nViewing Variable as Tensor: \\n\", tf.convert_to_tensor(my_var))\n",
"print(\"\\nIndex of highest value: \\n\", tf.argmax(my_var))\n",
"\n",
"# This creates a new tensor; it does not reshape the variable.\n",
"print(\"\\nCopying and reshaping: \", tf.reshape(my_var, ([1,4])))"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"\n",
"Viewing Variable as Tensor: \n",
" tf.Tensor(\n",
"[[1. 2.]\n",
" [3. 4.]], shape=(2, 2), dtype=float32)\n",
"\n",
"Index of highest value: \n",
" tf.Tensor([1 1], shape=(2,), dtype=int64)\n",
"\n",
"Copying and reshaping: tf.Tensor([[1. 2. 3. 4.]], shape=(1, 4), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zIg02e-pbESj",
"colab_type": "text"
},
"source": [
"Variables are backed by tensors. We can reassign the tensor using `tf.Variable.assign`. Calling assign does not (usually) allocate a new tensor; instead, the existing tensor's memory is reused."
]
},
{
"cell_type": "code",
"metadata": {
"id": "vo3gtTTdaFkK",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "b21c65d7-acdf-4dea-f09b-a73ee6fe417c"
},
"source": [
"a = tf.Variable([2.0, 3.0])\n",
"\n",
"# This will keep the same dtype, float32\n",
"# Assigning is successful since shape is same\n",
"a.assign([1, 2])\n",
"a"
],
"execution_count": 10,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tf.Variable 'Variable:0' shape=(2,) dtype=float32, numpy=array([1., 2.], dtype=float32)>"
]
},
"metadata": {
"tags": []
},
"execution_count": 10
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "RsMCrevCbvTy",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "c3dda58c-5121-4cf7-c373-6171b7e6d6b0"
},
"source": [
"# Not allowed as it resizes the variable: \n",
"try:\n",
" a.assign([1.0, 2.0, 3.0])\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")"
],
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"text": [
"ValueError: Shapes (2,) and (3,) are incompatible\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GiEA_KE4mR4a",
"colab_type": "text"
},
"source": [
"When we use a variable like a tensor in operations, we should usually operate on the backing tensor.\n",
"\n",
"Creating new variables from existing variables duplicates the backing tensors. Two variables will not share the same memory."
]
},
{
"cell_type": "code",
"metadata": {
"id": "rX0AgqoVmegJ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 100
},
"outputId": "e5dacc37-ab10-4b2e-a17a-d2d769f6128f"
},
"source": [
"a = tf.Variable([2.0, 3.0])\n",
"\n",
"# Create b based on the value of a\n",
"b = tf.Variable(a)\n",
"a.assign([5, 6])\n",
"\n",
"# values in a and b might be same but memory address is different\n",
"print(a.numpy())\n",
"print(b.numpy())\n",
"\n",
"print()\n",
"\n",
"# other versions of assign\n",
"print(a.assign_add([2,3]).numpy())\n",
"print(a.assign_sub([7,9]).numpy())\n"
],
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"text": [
"[5. 6.]\n",
"[2. 3.]\n",
"\n",
"[7. 9.]\n",
"[0. 0.]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jA7CfRe6nG9-",
"colab_type": "text"
},
"source": [
"## Lifecycles, naming, and watching\n",
"\n",
"In TensorFlow, `tf.Variable` instance have the same lifecycle as other Python objects. When there are no references to a variable it is automatically deallocated.\n",
"\n",
"Variables can also be named which can help us to track and debug them. We can give two variables the same name."
]
},
{
"cell_type": "code",
"metadata": {
"id": "hx_AzCC4nV-5",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 183
},
"outputId": "f4a539c6-f4ab-40c9-b99d-38131e8f409f"
},
"source": [
"# Create a and b; they have the same value but are backed by different tensors.\n",
"a = tf.Variable(my_tensor, name=\"Mark\")\n",
"\n",
"# A new variable with the same name, but different value\n",
"# Note that the scalar operations like +, -, * or / are broadcast\n",
"b = tf.Variable(my_tensor ** 2, name=\"Mark\")\n",
"\n",
"print(a)\n",
"print(b)\n",
"\n",
"print()\n",
"\n",
"# Elementwise-unequal, despite having the same name\n",
"print(a == b)"
],
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"text": [
"<tf.Variable 'Mark:0' shape=(2, 2) dtype=float32, numpy=\n",
"array([[1., 2.],\n",
" [3., 4.]], dtype=float32)>\n",
"<tf.Variable 'Mark:0' shape=(2, 2) dtype=float32, numpy=\n",
"array([[ 1., 4.],\n",
" [ 9., 16.]], dtype=float32)>\n",
"\n",
"tf.Tensor(\n",
"[[ True False]\n",
" [False False]], shape=(2, 2), dtype=bool)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "awP_HcJ8pDBW",
"colab_type": "text"
},
"source": [
"Variable names are preserved when saving and loading models. By default, variables in models will acquire unique variable names automatically, so we don't need to assign them ourself unless we want to.\n",
"\n",
"Although variables are important for differentiation, some variables will not need to be differentiated. We can turn off gradients for a variable by setting `trainable` to false at creation. \n",
"\n",
"An example of a variable that would not need gradients is a training step counter"
]
},
{
"cell_type": "code",
"metadata": {
"id": "rB3ADZchpN1Q",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "8e7a8557-1dba-4787-b434-b67518769068"
},
"source": [
"step_counter = tf.Variable(1, trainable=False)\n",
"step_counter"
],
"execution_count": 14,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tf.Variable 'Variable:0' shape=() dtype=int32, numpy=1>"
]
},
"metadata": {
"tags": []
},
"execution_count": 14
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OFruI9-VpezL",
"colab_type": "text"
},
"source": [
"## Placing variables and tensors\n",
"\n",
"For better performance, TensorFlow will attempt to place tensors and variables on the fastest device compatible with its dtype. This means most variables are placed on a GPU if one is available."
]
},
{
"cell_type": "code",
"metadata": {
"id": "VG85AHZOp5cf",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "26f13e60-5bee-456c-85d0-5dc3be496c59"
},
"source": [
"# with tf.device('CPU:0'):\n",
"with tf.device('GPU:0'):\n",
"\n",
" # Create some tensors\n",
" a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n",
" b = tf.constant([[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]])\n",
" c = tf.matmul(a, b)\n",
"\n",
"print(c)"
],
"execution_count": 15,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[22. 28.]\n",
" [49. 64.]], shape=(2, 2), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4MMymJNGqxJu",
"colab_type": "text"
},
"source": [
"However, we can override this. We can place a float tensor and a variable on the CPU, even if a GPU is available. By turning on device placement logging, we can see where the variable is placed.\n",
"\n",
"It's possible to set the location of a variable or tensor on one device and do the computation on another device. This will introduce delay, as data needs to be copied between the devices."
]
},
{
"cell_type": "code",
"metadata": {
"id": "KZql_DfPrHJb",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "816acf2e-abec-46df-f74c-201386c9212e"
},
"source": [
"with tf.device('CPU:0'):\n",
" a = tf.Variable([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]])\n",
" b = tf.Variable([[1.0, 2.0, 3.0]])\n",
"\n",
"with tf.device('GPU:0'):\n",
" # Element-wise multiply\n",
" k = a * b\n",
"\n",
"print(k)"
],
"execution_count": 16,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[ 1. 4. 9.]\n",
" [ 4. 10. 18.]], shape=(2, 3), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2ECFtOKmXV9H",
"colab_type": "text"
},
"source": [
"## Rank/Degree of a Tensor\n",
"\n",
"This simply means number of dimensions involved in a tensor. \n",
"\n",
"**Note:** A tensor of rank 0 is known as scalar tensor"
]
},
{
"cell_type": "code",
"metadata": {
"id": "Kri2I1maW4K3",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "7099a509-1fdf-4990-d415-d2f7e6cc5b5c"
},
"source": [
"rank_0_tensor = tf.constant(4)\n",
"print(rank_0_tensor)"
],
"execution_count": 17,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(4, shape=(), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "61GnKe2IX3S-",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "407cca26-a679-4fd3-e452-73b3564277b5"
},
"source": [
"rank_1_tensor = tf.constant([2.0, 3.0, 4.0])\n",
"print(rank_1_tensor)"
],
"execution_count": 18,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([2. 3. 4.], shape=(3,), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "YzRLzDmYX5n4",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 83
},
"outputId": "a5dab959-a6c2-45c6-88c3-0f1025f80a92"
},
"source": [
"rank_2_tensor = tf.constant([[1, 2],\n",
" [3, 4],\n",
" [5, 6]], dtype=tf.float16)\n",
"print(rank_2_tensor)"
],
"execution_count": 19,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[1. 2.]\n",
" [3. 4.]\n",
" [5. 6.]], shape=(3, 2), dtype=float16)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "yVC0MrGhbpH7",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 166
},
"outputId": "9631bcfd-a63b-4afc-dfd9-ddc75ff3329a"
},
"source": [
"# There can be an arbitrary number of axes (sometimes called \"dimensions\")\n",
"rank_3_tensor = tf.constant([\n",
" [[0, 1, 2, 3, 4],\n",
" [5, 6, 7, 8, 9]],\n",
" [[10, 11, 12, 13, 14],\n",
" [15, 16, 17, 18, 19]],\n",
" [[20, 21, 22, 23, 24],\n",
" [25, 26, 27, 28, 29]],])\n",
" \n",
"print(rank_3_tensor)"
],
"execution_count": 20,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[[ 0 1 2 3 4]\n",
" [ 5 6 7 8 9]]\n",
"\n",
" [[10 11 12 13 14]\n",
" [15 16 17 18 19]]\n",
"\n",
" [[20 21 22 23 24]\n",
" [25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zgeSDcwcr-Lp",
"colab_type": "text"
},
"source": [
"We can convert a tensor to a NumPy array either using `np.array` or the `tensor.numpy` method"
]
},
{
"cell_type": "code",
"metadata": {
"id": "l5M_Y7j7r0gf",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "49e94515-d129-4ea7-e104-923665d112b5"
},
"source": [
"np.array(rank_2_tensor)"
],
"execution_count": 21,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[1., 2.],\n",
" [3., 4.],\n",
" [5., 6.]], dtype=float16)"
]
},
"metadata": {
"tags": []
},
"execution_count": 21
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "IYUWzAhesIDY",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "2d8f9e9a-6ed2-4d58-e305-dde1539f1d68"
},
"source": [
"rank_2_tensor.numpy()"
],
"execution_count": 22,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"array([[1., 2.],\n",
" [3., 4.],\n",
" [5., 6.]], dtype=float16)"
]
},
"metadata": {
"tags": []
},
"execution_count": 22
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9s7BICRLsZyv",
"colab_type": "text"
},
"source": [
"The base `tf.Tensor` class requires tensors to be \"rectangular\"---that is, along each axis, every element is the same size. However, there are specialized types of Tensors that can handle different shapes:\n",
"\n",
" - ragged (see RaggedTensor below)\n",
" - sparse (see SparseTensor below)\n",
"\n",
"We can do basic math on tensors, including addition, element-wise multiplication, and matrix multiplication."
]
},
{
"cell_type": "code",
"metadata": {
"id": "q8SCu8UhsSTf",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 216
},
"outputId": "9ca20654-da03-4e41-9434-de412ca26c12"
},
"source": [
"a = tf.constant([[1, 2],\n",
" [3, 4]])\n",
"b = tf.constant([[1, 1],\n",
" [1, 1]])\n",
"\n",
"print(tf.add(a, b), \"\\n\")\n",
"print(tf.multiply(a, b), \"\\n\")\n",
"print(tf.matmul(a, b), \"\\n\")"
],
"execution_count": 23,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[2 3]\n",
" [4 5]], shape=(2, 2), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[1 2]\n",
" [3 4]], shape=(2, 2), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[3 3]\n",
" [7 7]], shape=(2, 2), dtype=int32) \n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "qRbavpSOsxFA",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 216
},
"outputId": "5abc35fe-6bc0-46bb-f6d2-7e15abea7f7b"
},
"source": [
"print(a + b, \"\\n\") # element-wise addition\n",
"print(a * b, \"\\n\") # element-wise multiplication\n",
"print(a @ b, \"\\n\") # matrix multiplication"
],
"execution_count": 24,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[2 3]\n",
" [4 5]], shape=(2, 2), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[1 2]\n",
" [3 4]], shape=(2, 2), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[3 3]\n",
" [7 7]], shape=(2, 2), dtype=int32) \n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6mXVX2tztCQE",
"colab_type": "text"
},
"source": [
"Tensors are used in all kinds of operations."
]
},
{
"cell_type": "code",
"metadata": {
"id": "xiuhGSBJs0bU",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 100
},
"outputId": "cd38aeff-0a37-44d1-dde5-bc7e5c6375d0"
},
"source": [
"c = tf.constant([[4.0, 5.0], [10.0, 1.0]])\n",
"\n",
"# Find the largest value\n",
"print(tf.reduce_max(c))\n",
"# Find the index of the largest value\n",
"print(tf.argmax(c))\n",
"# Compute the softmax\n",
"print(tf.nn.softmax(c))"
],
"execution_count": 25,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(10.0, shape=(), dtype=float32)\n",
"tf.Tensor([1 0], shape=(2,), dtype=int64)\n",
"tf.Tensor(\n",
"[[2.6894143e-01 7.3105860e-01]\n",
" [9.9987662e-01 1.2339458e-04]], shape=(2, 2), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JfazDoZptKPt",
"colab_type": "text"
},
"source": [
"## Tensor Shapes\n",
"\n",
"Tensors have shapes. Some vocabulary:\n",
"\n",
" - `Shape`: The length (number of elements) of each of the dimensions of a tensor.\n",
" - `Rank`: Number of tensor dimensions. A scalar has rank 0, a vector has rank 1, a matrix is rank 2.\n",
" - `Axis` or `Dimension`: A particular dimension of a tensor.\n",
" - `Size`: The total number of items in the tensor, the product shape vector"
]
},
{
"cell_type": "code",
"metadata": {
"id": "NTqRarNatF4j",
"colab_type": "code",
"colab": {}
},
"source": [
"rank_4_tensor = tf.zeros([3, 2, 4, 5])"
],
"execution_count": 26,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "hMpWFT1BuaVt",
"colab_type": "text"
},
"source": [
"![rank-4-tensor](https://user-images.githubusercontent.com/53504602/88804798-0b386400-d1cc-11ea-921f-10da4bee975f.png)\n",
"<center> <img src=\"https://user-images.githubusercontent.com/53504602/88805060-52bef000-d1cc-11ea-871a-56a463d387dc.png\"> </center>"
]
},
{
"cell_type": "code",
"metadata": {
"id": "g81qP9IAtZgC",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 216
},
"outputId": "fe1f71a2-458d-4171-e130-b0e5fa19926c"
},
"source": [
"print(\"\\nType of every element:\", rank_4_tensor.dtype)\n",
"print(\"\\nNumber of dimensions:\", rank_4_tensor.ndim)\n",
"print(\"\\nShape of tensor:\", rank_4_tensor.shape)\n",
"print(\"\\nElements along axis 0 of tensor:\", rank_4_tensor.shape[0])\n",
"print(\"\\nElements along the last axis of tensor:\", rank_4_tensor.shape[-1])\n",
"print(\"\\nTotal number of elements (3*2*4*5): \", tf.size(rank_4_tensor).numpy())"
],
"execution_count": 27,
"outputs": [
{
"output_type": "stream",
"text": [
"\n",
"Type of every element: <dtype: 'float32'>\n",
"\n",
"Number of dimensions: 4\n",
"\n",
"Shape of tensor: (3, 2, 4, 5)\n",
"\n",
"Elements along axis 0 of tensor: 3\n",
"\n",
"Elements along the last axis of tensor: 5\n",
"\n",
"Total number of elements (3*2*4*5): 120\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7X1Z-ep9vTMK",
"colab_type": "text"
},
"source": [
"## Indexing\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vh4uX3RAwNON",
"colab_type": "text"
},
"source": [
"\n",
"### Single-axis indexing\n",
"\n",
"TensorFlow follow standard python indexing rules, similar to indexing a list or a string in python, and the bacic rules for numpy indexing.\n",
"\n",
" - indexes start at 0\n",
" - negative indices count backwards from the end\n",
" - colons, `:`, are used for slices `start:stop:step`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "l1VLrObctgZH",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 83
},
"outputId": "5480eaad-5454-42ed-991c-c438fcc3ca99"
},
"source": [
"rank_1_tensor = tf.constant([0, 1, 1, 2, 3, 5, 8, 13, 21, 34])\n",
"print(rank_1_tensor.numpy())\n",
"\n",
"# Indexing with a scalar removes the dimension\n",
"print(\"First:\", rank_1_tensor[0].numpy())\n",
"print(\"Second:\", rank_1_tensor[1].numpy())\n",
"print(\"Last:\", rank_1_tensor[-1].numpy())"
],
"execution_count": 28,
"outputs": [
{
"output_type": "stream",
"text": [
"[ 0 1 1 2 3 5 8 13 21 34]\n",
"First: 0\n",
"Second: 1\n",
"Last: 34\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "A7tmaCdev-RG",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 116
},
"outputId": "a699562c-72fc-412f-dc0f-0444ae504264"
},
"source": [
"# Indexing with a : slice keeps the dimension\n",
"print(\"Everything:\", rank_1_tensor[:].numpy())\n",
"print(\"Before 4:\", rank_1_tensor[:4].numpy())\n",
"print(\"From 4 to the end:\", rank_1_tensor[4:].numpy())\n",
"print(\"From 2, before 7:\", rank_1_tensor[2:7].numpy())\n",
"print(\"Every other item:\", rank_1_tensor[::2].numpy())\n",
"print(\"Reversed:\", rank_1_tensor[::-1].numpy())"
],
"execution_count": 29,
"outputs": [
{
"output_type": "stream",
"text": [
"Everything: [ 0 1 1 2 3 5 8 13 21 34]\n",
"Before 4: [0 1 1 2]\n",
"From 4 to the end: [ 3 5 8 13 21 34]\n",
"From 2, before 7: [1 2 3 5 8]\n",
"Every other item: [ 0 1 3 8 21]\n",
"Reversed: [34 21 13 8 5 3 2 1 1 0]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5YIKuBrowJGl",
"colab_type": "text"
},
"source": [
"### Multi-axis indexing\n",
"\n",
"Higher rank tensors are indexed by passing multiple indices.\n",
"\n",
"The single-axis exact same rules as in the single-axis case apply to each axis independently."
]
},
{
"cell_type": "code",
"metadata": {
"id": "HwX5M-9pwCrW",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 150
},
"outputId": "5576f636-cb34-4d34-f369-d1278b9b916d"
},
"source": [
"print(rank_3_tensor.numpy())"
],
"execution_count": 30,
"outputs": [
{
"output_type": "stream",
"text": [
"[[[ 0 1 2 3 4]\n",
" [ 5 6 7 8 9]]\n",
"\n",
" [[10 11 12 13 14]\n",
" [15 16 17 18 19]]\n",
"\n",
" [[20 21 22 23 24]\n",
" [25 26 27 28 29]]]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "slTishmMwR0a",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "3be4e10f-74fa-4bd3-b657-45ef367dc1a7"
},
"source": [
"# Pull out a 2nd row from 2nd batch a 3-rank tensor\n",
"print(rank_3_tensor[1, 1].numpy())"
],
"execution_count": 31,
"outputs": [
{
"output_type": "stream",
"text": [
"[15 16 17 18 19]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "2jr2fjeYwVv5",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "a38f40ca-6fc7-49e8-e611-89f9f2b5d74a"
},
"source": [
"# Pull out a 2nd value from 2nd row from 2nd batch of a 3-rank tensor\n",
"print(rank_3_tensor[1, 1][1].numpy())"
],
"execution_count": 32,
"outputs": [
{
"output_type": "stream",
"text": [
"16\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "m3v0fiNJxLEH",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 83
},
"outputId": "63251a50-d64a-45cb-c812-712263e16972"
},
"source": [
"print(rank_2_tensor)"
],
"execution_count": 33,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[1. 2.]\n",
" [3. 4.]\n",
" [5. 6.]], shape=(3, 2), dtype=float16)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "JS5fwlqhwmHF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 150
},
"outputId": "7be47527-86f1-434e-beb4-fb85964b39ba"
},
"source": [
"# Get row and column tensors\n",
"print(\"Second row:\", rank_2_tensor[1, :].numpy())\n",
"print(\"Second column:\", rank_2_tensor[:, 1].numpy())\n",
"print(\"Last row:\", rank_2_tensor[-1, :].numpy())\n",
"print(\"First item in last column:\", rank_2_tensor[0, -1].numpy())\n",
"print(\"Skip the first row:\")\n",
"print(rank_2_tensor[1:, :].numpy(), \"\\n\")"
],
"execution_count": 34,
"outputs": [
{
"output_type": "stream",
"text": [
"Second row: [3. 4.]\n",
"Second column: [2. 4. 6.]\n",
"Last row: [5. 6.]\n",
"First item in last column: 2.0\n",
"Skip the first row:\n",
"[[3. 4.]\n",
" [5. 6.]] \n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SEp0lPJhyK6e",
"colab_type": "text"
},
"source": [
"## Manipulating Shapes\n",
"\n",
"Reshaping a tensor is of great utility.\n",
"\n",
"The `tf.reshape` operation is fast and cheap as the underlying data does not need to be duplicated."
]
},
{
"cell_type": "code",
"metadata": {
"id": "B1HAwcRnx1dg",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "55c058fe-865a-4a6b-ee6b-902d25157ac7"
},
"source": [
"# Shape returns a `TensorShape` object that shows the size on each dimension\n",
"var_x = tf.Variable(tf.constant([[1], [2], [3]]))\n",
"print(var_x.shape)"
],
"execution_count": 35,
"outputs": [
{
"output_type": "stream",
"text": [
"(3, 1)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "UiCWod2qyWtM",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "690c342d-da09-43fc-ea9f-88d7ce0dc93a"
},
"source": [
"type(var_x)"
],
"execution_count": 36,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"tensorflow.python.ops.resource_variable_ops.ResourceVariable"
]
},
"metadata": {
"tags": []
},
"execution_count": 36
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "9j75D1bYybvp",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "5e0f7553-666b-4ca3-f2ea-aaf6d56de39d"
},
"source": [
"# We can convert this object into a Python list, too\n",
"print(var_x.shape.as_list())"
],
"execution_count": 37,
"outputs": [
{
"output_type": "stream",
"text": [
"[3, 1]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FHzsAtwQyvQs",
"colab_type": "text"
},
"source": [
"Reshaping is fast and cheap as the underlying data does not need to be duplicated."
]
},
{
"cell_type": "code",
"metadata": {
"id": "nqxRtSIZylMZ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "3fb1eec4-3c04-4e55-b293-a9584d6ca065"
},
"source": [
"# We can reshape a tensor to a new shape.\n",
"# Note that we're passing in a list\n",
"reshaped = tf.reshape(var_x, [1, 3])\n",
"\n",
"print(var_x.shape)\n",
"print(reshaped.shape)"
],
"execution_count": 38,
"outputs": [
{
"output_type": "stream",
"text": [
"(3, 1)\n",
"(1, 3)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3n2K3HoAy6iy",
"colab_type": "text"
},
"source": [
"The data maintains it's layout in memory and a new tensor is created, with the requested shape, pointing to the same data. TensorFlow uses C-style \"row-major\" memory ordering, where incrementing the right-most index corresponds to a single step in memory."
]
},
{
"cell_type": "code",
"metadata": {
"id": "D5fWmZlyyzD6",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 416
},
"outputId": "300943e6-0d04-4679-de40-e38f9a36c2c3"
},
"source": [
"print(rank_3_tensor)\n",
"\n",
"# A `-1` passed in the `shape` argument says \"Whatever fits\".\n",
"print(tf.reshape(rank_3_tensor, [-1]))\n",
"\n",
"print(tf.reshape(rank_3_tensor, [3*2, 5]), \"\\n\")\n",
"print(tf.reshape(rank_3_tensor, [3, -1]))"
],
"execution_count": 39,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[[ 0 1 2 3 4]\n",
" [ 5 6 7 8 9]]\n",
"\n",
" [[10 11 12 13 14]\n",
" [15 16 17 18 19]]\n",
"\n",
" [[20 21 22 23 24]\n",
" [25 26 27 28 29]]], shape=(3, 2, 5), dtype=int32)\n",
"tf.Tensor(\n",
"[ 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23\n",
" 24 25 26 27 28 29], shape=(30,), dtype=int32)\n",
"tf.Tensor(\n",
"[[ 0 1 2 3 4]\n",
" [ 5 6 7 8 9]\n",
" [10 11 12 13 14]\n",
" [15 16 17 18 19]\n",
" [20 21 22 23 24]\n",
" [25 26 27 28 29]], shape=(6, 5), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[ 0 1 2 3 4 5 6 7 8 9]\n",
" [10 11 12 13 14 15 16 17 18 19]\n",
" [20 21 22 23 24 25 26 27 28 29]], shape=(3, 10), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9sa_4J3yzkSq",
"colab_type": "text"
},
"source": [
"Swapping axes in `tf.reshape` does not work, you need `tf.transpose` for that. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "2c4CWu7pzPqV",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"outputId": "b0902080-330b-499f-a63f-dcf05c0c9abb"
},
"source": [
"# We can't reorder axes with reshape.\n",
"print(tf.reshape(rank_3_tensor, [2, 3, 5]), \"\\n\") \n",
"\n",
"# This is a mess\n",
"print(tf.reshape(rank_3_tensor, [5, 6]), \"\\n\")\n",
"\n",
"# This doesn't work at all\n",
"try:\n",
" tf.reshape(rank_3_tensor, [7, -1])\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")"
],
"execution_count": 40,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[[ 0 1 2 3 4]\n",
" [ 5 6 7 8 9]\n",
" [10 11 12 13 14]]\n",
"\n",
" [[15 16 17 18 19]\n",
" [20 21 22 23 24]\n",
" [25 26 27 28 29]]], shape=(2, 3, 5), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[ 0 1 2 3 4 5]\n",
" [ 6 7 8 9 10 11]\n",
" [12 13 14 15 16 17]\n",
" [18 19 20 21 22 23]\n",
" [24 25 26 27 28 29]], shape=(5, 6), dtype=int32) \n",
"\n",
"InvalidArgumentError: Input to reshape is a tensor with 30 values, but the requested shape requires a multiple of 7 [Op:Reshape]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "hVEr-zzP0J4U",
"colab_type": "text"
},
"source": [
"![rank-3-tensor-reordering](https://user-images.githubusercontent.com/53504602/88807586-88190d00-d1cf-11ea-9307-cc3f8084339a.png)\n",
"\n",
"Here we see that we were able to reshape the tensor into (2, 3, 5) by swapping the axes, but couldn't maintain the ordering of values"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rvVNLCwN32DI",
"colab_type": "text"
},
"source": [
"## More on DTypes\n",
"\n",
"To inspect a `tf.Tensor`'s data type use the `Tensor.dtype` property.\n",
"\n",
"When creating a `tf.Tensor` from a Python object you may optionally specify the datatype.\n",
"\n",
"If we don't, TensorFlow chooses a datatype that can represent our data. TensorFlow converts Python integers to `tf.int32` and python floating point numbers to `tf.float32`. Otherwise TensorFlow uses the same rules NumPy uses when converting to arrays.\n",
"\n",
"We can cast from type to type."
]
},
{
"cell_type": "code",
"metadata": {
"id": "Wqz9NS7azuMn",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "fcd78d1f-62f8-486e-ec64-c581355819f1"
},
"source": [
"f64_tensor = tf.constant([2.2, 3.3, 4.4], dtype=tf.float64)\n",
"f16_tensor = tf.cast(f64_tensor, dtype=tf.float16)\n",
"\n",
"# Now, let's cast to an uint8 and lose the decimal precision\n",
"u8_tensor = tf.cast(f16_tensor, dtype=tf.uint8)\n",
"print(u8_tensor)"
],
"execution_count": 41,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([2 3 4], shape=(3,), dtype=uint8)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FSvLaWuR4PSC",
"colab_type": "text"
},
"source": [
"## Broadcasting\n",
"\n",
"Broadcasting is a concept borrowed from the equivalent feature in NumPy. In short, under certain conditions, smaller tensors are \"stretched\" automatically to fit larger tensors when running combined operations on them.\n",
"\n",
"The simplest and most common case is when you attempt to multiply or add a tensor to a scalar. In that case, the scalar is broadcast to be the same shape as the other argument."
]
},
{
"cell_type": "code",
"metadata": {
"id": "YyKrFa5x4KEb",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "a5495ec5-ec7e-4557-c9a6-916abfb7a1c6"
},
"source": [
"x = tf.constant([1, 2, 3])\n",
"\n",
"y = tf.constant(2)\n",
"z = tf.constant([2, 2, 2])\n",
"# All of these are the same computation\n",
"print(tf.multiply(x, 2))\n",
"print(x * y)\n",
"print(x * z)"
],
"execution_count": 42,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([2 4 6], shape=(3,), dtype=int32)\n",
"tf.Tensor([2 4 6], shape=(3,), dtype=int32)\n",
"tf.Tensor([2 4 6], shape=(3,), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "p-_yEAfs4_Fr",
"colab_type": "text"
},
"source": [
"Likewise, 1-sized dimensions can be stretched out to match the other arguments. Both arguments can be stretched in the same computation.\n",
"\n",
"In this case a 3x1 matrix is element-wise multiplied by a 1x4 matrix to produce a 3x4 matrix. Note how the leading 1 is optional: The shape of y is `[4]`."
]
},
{
"cell_type": "code",
"metadata": {
"id": "wsOahbW-40AT",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 200
},
"outputId": "4607535d-7af9-4dda-c5ab-f7b3380d1fc2"
},
"source": [
"# These are the same computations\n",
"x = tf.reshape(x,[3,1])\n",
"y = tf.range(1, 5)\n",
"print(x, \"\\n\")\n",
"print(y, \"\\n\")\n",
"print(tf.multiply(x, y))"
],
"execution_count": 43,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[1]\n",
" [2]\n",
" [3]], shape=(3, 1), dtype=int32) \n",
"\n",
"tf.Tensor([1 2 3 4], shape=(4,), dtype=int32) \n",
"\n",
"tf.Tensor(\n",
"[[ 1 2 3 4]\n",
" [ 2 4 6 8]\n",
" [ 3 6 9 12]], shape=(3, 4), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ndRbyaUq5p0T",
"colab_type": "text"
},
"source": [
"![broadcasting](https://user-images.githubusercontent.com/53504602/88810398-eb586e80-d1d2-11ea-8b48-1deb5f402783.png)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "aAsaaSoi5L3B",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 83
},
"outputId": "1b117153-c83b-43e4-c387-4a9964b9afec"
},
"source": [
"x_stretch = tf.constant([[1, 1, 1, 1],\n",
" [2, 2, 2, 2],\n",
" [3, 3, 3, 3]])\n",
"\n",
"y_stretch = tf.constant([[1, 2, 3, 4],\n",
" [1, 2, 3, 4],\n",
" [1, 2, 3, 4]])\n",
"\n",
"print(x_stretch * y_stretch)"
],
"execution_count": 44,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[ 1 2 3 4]\n",
" [ 2 4 6 8]\n",
" [ 3 6 9 12]], shape=(3, 4), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ciVjIcjV6BzD",
"colab_type": "text"
},
"source": [
"we can also do broadcasting using `tf.broadcast_to`\n",
"\n",
"`tf.broadcast_to` does nothing special to save memory. Here, we are materializing the tensor."
]
},
{
"cell_type": "code",
"metadata": {
"id": "UwW1m04C55AA",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 83
},
"outputId": "3b09c477-a594-480d-8324-3b9aadca3996"
},
"source": [
"print(tf.broadcast_to(tf.constant([1, 2, 3]), [3, 3]))"
],
"execution_count": 45,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(\n",
"[[1 2 3]\n",
" [1 2 3]\n",
" [1 2 3]], shape=(3, 3), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "El7ffoa_6iyB",
"colab_type": "text"
},
"source": [
"## `tf.convert_to_tensor`\n",
"\n",
"Most ops, like `tf.matmul` and `tf.reshape` take arguments of class `tf.Tensor`. However, in the above case, we frequently pass Python objects shaped like tensors.\n",
"\n",
"Most, but not all, operations call `convert_to_tensor` on non-tensor arguments. There is a registry of conversions, and most object classes like NumPy's `ndarray`, `TensorShape`, Python lists, and `tf.Variable` will all convert automatically.\n",
"\n",
"Use `tf.register_tensor_conversion_function`, and if we have our own type we'd like to automatically convert to a tensor."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zxATPa5G7BFs",
"colab_type": "text"
},
"source": [
"## Ragged Tensors\n",
"\n",
"A tensor with variable numbers of elements along some axis is called \"ragged\". Use `tf.ragged.RaggedTensor` for ragged data.\n",
"\n",
"For example, This cannot be represented as a regular tensor:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "aLDcaYo26OPX",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "fb22ad45-d703-4a02-ecba-9472b33f1cee"
},
"source": [
"ragged_list = [\n",
" [0, 1, 2, 3],\n",
" [4, 5],\n",
" [6, 7, 8],\n",
" [9]]\n",
"\n",
"try:\n",
" tensor = tf.constant(ragged_list)\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")"
],
"execution_count": 46,
"outputs": [
{
"output_type": "stream",
"text": [
"ValueError: Can't convert non-rectangular Python sequence to Tensor.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HP9Dl9bC_up6",
"colab_type": "text"
},
"source": [
"Instead create a `tf.RaggedTensor` using `tf.ragged.constant`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "N1D0m3id_qy5",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "135e8cda-396c-4573-ff72-a8e0005a6ac4"
},
"source": [
"ragged_tensor = tf.ragged.constant(ragged_list)\n",
"print(ragged_tensor)"
],
"execution_count": 47,
"outputs": [
{
"output_type": "stream",
"text": [
"<tf.RaggedTensor [[0, 1, 2, 3], [4, 5], [6, 7, 8], [9]]>\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "nVMP-yQC_ztc",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "557a70ec-7103-49da-cca2-47e526132557"
},
"source": [
"# shape of a tf.RaggedTensor contains unknown dimensions\n",
"print(ragged_tensor.shape)"
],
"execution_count": 48,
"outputs": [
{
"output_type": "stream",
"text": [
"(4, None)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "is29pqhcACFW",
"colab_type": "text"
},
"source": [
"## String tensors\n",
"\n",
"`tf.string` is a `dtype`, which is to say we can represent data as strings (variable-length byte arrays) in tensors.\n",
"\n",
"The strings are atomic and cannot be indexed the way Python strings are. The length of the string is not one of the dimensions of the tensor.\n",
"\n",
"Here is a scalar string tensor:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "6fg2Q1sb_9IQ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "3f9f1d22-ed33-4dc8-b855-337d8b23d283"
},
"source": [
"# Tensors can be strings, too here is a scalar string.\n",
"scalar_string_tensor = tf.constant(\"Gray wolf\")\n",
"print(scalar_string_tensor)"
],
"execution_count": 49,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(b'Gray wolf', shape=(), dtype=string)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "R6iDUykGANzj",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "fba08646-2d34-4383-b49d-36282aee2220"
},
"source": [
"# three string tensors of different lengths, this is OK.\n",
"tensor_of_strings = tf.constant([\"Gray wolf\",\n",
" \"Quick brown fox\",\n",
" \"Lazy dog\"])\n",
"\n",
"# Note that the shape is (3,). The string length is not included.\n",
"print(tensor_of_strings)"
],
"execution_count": 50,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([b'Gray wolf' b'Quick brown fox' b'Lazy dog'], shape=(3,), dtype=string)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qL4lkx7KAZj0",
"colab_type": "text"
},
"source": [
"In the above printout the `b` prefix indicates that `tf.string` dtype is not a unicode string, but a byte-string.\n",
"\n",
"If we pass unicode characters they are utf-8 encoded."
]
},
{
"cell_type": "code",
"metadata": {
"id": "ZorYTCozAUY3",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "976a2b4e-d5d1-4b5c-d03b-e1ed81f80571"
},
"source": [
"tf.constant(\"🥳👍\")"
],
"execution_count": 51,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tf.Tensor: shape=(), dtype=string, numpy=b'\\xf0\\x9f\\xa5\\xb3\\xf0\\x9f\\x91\\x8d'>"
]
},
"metadata": {
"tags": []
},
"execution_count": 51
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cqcac_S_Alrc",
"colab_type": "text"
},
"source": [
"Some basic functions with strings can be found in `tf.strings`, including `tf.strings.split`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "XtOpHL7jAihj",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "dccd99e9-ffe2-4cf7-def2-eeebb55226e6"
},
"source": [
"# We can use split to split a string into a set of tensors\n",
"print(tf.strings.split(scalar_string_tensor, sep=\" \"))"
],
"execution_count": 52,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([b'Gray' b'wolf'], shape=(2,), dtype=string)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "Tbpvyl__Aqgl",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "497e4809-216f-4431-e784-7fa96966df9a"
},
"source": [
"# because each string might be separated into many different parts\n",
"# we get a RaggedTensor\n",
"print(tf.strings.split(tensor_of_strings))"
],
"execution_count": 53,
"outputs": [
{
"output_type": "stream",
"text": [
"<tf.RaggedTensor [[b'Gray', b'wolf'], [b'Quick', b'brown', b'fox'], [b'Lazy', b'dog']]>\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "N6M3DxJUBCEo",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "416ca18f-176e-4637-f865-cafc7cba7095"
},
"source": [
"# convert a string into a floating tensor\n",
"text = tf.constant(\"1 10 100\")\n",
"print(tf.strings.to_number(tf.strings.split(text, \" \")))"
],
"execution_count": 54,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([ 1. 10. 100.], shape=(3,), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pJhYhDDQBVMq",
"colab_type": "text"
},
"source": [
"We can't use `tf.cast` to turn a string tensor into numbers, but we can convert it into bytes, and then into numbers."
]
},
{
"cell_type": "code",
"metadata": {
"id": "41E3rtFNBR0N",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "35ce211a-8122-4a4d-96ca-87887bdb0dd1"
},
"source": [
"byte_strings = tf.strings.bytes_split(tf.constant(\"Duck\"))\n",
"byte_ints = tf.io.decode_raw(tf.constant(\"Duck\"), tf.uint8)\n",
"print(\"Byte strings:\", byte_strings)\n",
"print(\"Bytes:\", byte_ints)"
],
"execution_count": 55,
"outputs": [
{
"output_type": "stream",
"text": [
"Byte strings: tf.Tensor([b'D' b'u' b'c' b'k'], shape=(4,), dtype=string)\n",
"Bytes: tf.Tensor([ 68 117 99 107], shape=(4,), dtype=uint8)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LO-h_5KwBkqv",
"colab_type": "text"
},
"source": [
"The `tf.string` dtype is used for all raw bytes data in TensorFlow. The `tf.io` module contains functions for converting data to and from bytes, including decoding images and parsing csv."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "G-SkL7ETBq6o",
"colab_type": "text"
},
"source": [
"## Sparse tensors\n",
"\n",
"Sometimes, our data is sparse, like a very wide embedding space. TensorFlow supports `tf.sparse.SparseTensor` and related operations to store sparse data efficiently."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SHzIWWjhDqyk",
"colab_type": "text"
},
"source": [
"![sparseTensor](https://user-images.githubusercontent.com/53504602/88815483-075f0e80-d1d9-11ea-9714-b2bc7ac278f0.png)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "L0dCQJx0BeyX",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 170
},
"outputId": "c6530722-2813-4dda-b19e-4bc500921ab1"
},
"source": [
"# Sparse tensors store values by index in a memory-efficient manner\n",
"sparse_tensor = tf.sparse.SparseTensor(indices=[[0, 0], [1, 2]],\n",
" values=[1, 2],\n",
" dense_shape=[3, 4])\n",
"print(sparse_tensor, \"\\n\")\n",
"\n",
"# We can convert sparse tensors to dense\n",
"print(tf.sparse.to_dense(sparse_tensor))"
],
"execution_count": 56,
"outputs": [
{
"output_type": "stream",
"text": [
"SparseTensor(indices=tf.Tensor(\n",
"[[0 0]\n",
" [1 2]], shape=(2, 2), dtype=int64), values=tf.Tensor([1 2], shape=(2,), dtype=int32), dense_shape=tf.Tensor([3 4], shape=(2,), dtype=int64)) \n",
"\n",
"tf.Tensor(\n",
"[[1 0 0 0]\n",
" [0 0 2 0]\n",
" [0 0 0 0]], shape=(3, 4), dtype=int32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "22uRy3N5HdeB",
"colab_type": "text"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mHv9tN0tEFrC",
"colab_type": "text"
},
"source": [
"# <h1 align=\"center\">-----Part - 2-----</h1>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9YxabPGKHy24",
"colab_type": "text"
},
"source": [
"## Gradients and Automatic Differentiation\n",
"\n",
"[Automatic differentiation](https://en.wikipedia.org/wiki/Automatic_differentiation) is useful for implementing machine learning algorithms such as [backpropagation](https://en.wikipedia.org/wiki/Backpropagation) for training neural networks."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JuZBm6VgIXJj",
"colab_type": "text"
},
"source": [
"### Computing gradients\n",
"\n",
"To differentiate automatically, TensorFlow needs to remember what operations happen in what order during the *forward pass*. Then, during the *backward pass*, TensorFlow traverses this list of operations in reverse order to compute gradients."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "DgHi8FqsI8mQ",
"colab_type": "text"
},
"source": [
"### Gradient tapes\n",
"\n",
"TensorFlow provides the `tf.GradientTape` API for automatic differentiation; that is, computing the gradient of a computation with respect to some inputs, usually `tf.Variables`. TensorFlow \"records\" relevant operations executed inside the context of a `tf.GradientTape` onto a \"tape\". TensorFlow then uses that tape to compute the gradients of a \"recorded\" computation using reverse mode differentiation."
]
},
{
"cell_type": "code",
"metadata": {
"id": "o5ezpS7vELU_",
"colab_type": "code",
"colab": {}
},
"source": [
"x = tf.Variable(3.0)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" y = x**2"
],
"execution_count": 57,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "HezrbCY1JlS3",
"colab_type": "text"
},
"source": [
"Once we've recorded some operations, `GradientTape.gradient(target, sources)` calculates the gradient of some target (often a loss) relative to some source (often the model's variables)."
]
},
{
"cell_type": "code",
"metadata": {
"id": "2d1LfXvBJbuf",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "c33cfa7d-e3f4-4070-b9bf-85e0662f5be4"
},
"source": [
"dy_dx = tape.gradient(y, x)\n",
"\n",
"# dy/dx = 2x\n",
"# for x = 3.0, dy/dx = 6.0\n",
"dy_dx.numpy()"
],
"execution_count": 58,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"6.0"
]
},
"metadata": {
"tags": []
},
"execution_count": 58
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VYI9TwHzKEwl",
"colab_type": "text"
},
"source": [
"The above example uses scalars, but `tf.GradientTape` works as easily on any tensor:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "CArN7kiyKBC-",
"colab_type": "code",
"colab": {}
},
"source": [
"w = tf.Variable(tf.random.normal((3, 2)), name='w')\n",
"b = tf.Variable(tf.zeros(2, dtype=tf.float32), name='b')\n",
"x = [[1., 2., 3.]]\n",
"\n",
"with tf.GradientTape(persistent=True) as tape:\n",
" y = x @ w + b\n",
" loss = tf.reduce_mean(y**2)"
],
"execution_count": 59,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "2DvBVXOEKrpT",
"colab_type": "text"
},
"source": [
"To get the gradient of `y` with respect to both variables, we can pass both as sources to the gradient method. The tape is flexible about how sources are passed and will accept any nested combination of lists or dictionaries and return the gradient structured the same way (`tf.nest`)."
]
},
{
"cell_type": "code",
"metadata": {
"id": "xD0rhGNXKqiK",
"colab_type": "code",
"colab": {}
},
"source": [
"[dl_dw, dl_db] = tape.gradient(loss, [w, b])"
],
"execution_count": 60,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WCgCu_M7K5MI",
"colab_type": "text"
},
"source": [
"The gradient with respect to each source has the shape of the source:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2laVs2_WKzfs",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "f6fffc4c-563c-4de0-c32c-28ecd9a89b47"
},
"source": [
"print(w.shape)\n",
"print(dl_dw.shape)"
],
"execution_count": 61,
"outputs": [
{
"output_type": "stream",
"text": [
"(3, 2)\n",
"(3, 2)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "821lABq4K8rC",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "02fe3337-1ca2-463a-c88b-ca69603cfc27"
},
"source": [
"print(b.shape)\n",
"print(dl_db.shape)"
],
"execution_count": 62,
"outputs": [
{
"output_type": "stream",
"text": [
"(2,)\n",
"(2,)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JUIrVIHSLN9z",
"colab_type": "text"
},
"source": [
"## Gradients with respect to a model\n",
"\n",
"It's common to collect `tf.Variables` into a `tf.Module` or one of its subclasses (`layers.Layer`, `keras.Model`) for checkpointing and exporting.\n",
"\n",
"In most cases, we would like to calculate gradients with respect to a model's trainable variables. Since all subclasses of `tf.Module` aggregate their variables in the `Module.trainable_variables` property, we can calculate these gradients in a few lines of code:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "my7rOIUbLFEi",
"colab_type": "code",
"colab": {}
},
"source": [
"layer = tf.keras.layers.Dense(2, activation='sigmoid')\n",
"x = tf.constant([[1., 2., 3.]])\n",
"\n",
"with tf.GradientTape() as tape:\n",
" # Forward pass\n",
" y = layer(x)\n",
" loss = tf.reduce_mean(y**2)\n",
"\n",
"# Calculate gradients with respect to every trainable variable\n",
"grad = tape.gradient(loss, layer.trainable_variables)"
],
"execution_count": 63,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "dG2YYqaiLnON",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "7092a17a-114b-4f62-c876-39c3f4e3bca0"
},
"source": [
"for var, g in zip(layer.trainable_variables, grad):\n",
" print(f'{var.name}, shape: {g.shape}')"
],
"execution_count": 64,
"outputs": [
{
"output_type": "stream",
"text": [
"dense/kernel:0, shape: (3, 2)\n",
"dense/bias:0, shape: (2,)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "3AbYd9ruL7of",
"colab_type": "text"
},
"source": [
"## Controlling what the tape watches\n",
"\n",
"The default behavior is to record all operations after accessing a trainable `tf.Variable`. The reasons for this are:\n",
"\n",
" - The tape needs to know which operations to record in the forward pass to calculate the gradients in the backwards pass.\n",
" - The tape holds references to intermediate outputs, so you don't want to record unnecessary operations.\n",
" - The most common use case involves calculating the gradient of a loss with respect to all a model's trainable variables.\n",
"\n",
"For example the following fails to calculate a gradient because the `tf.Tensor` is not \"watched\" by default, and the `tf.Variable` is not trainable:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "fX3Q46rOL4NH",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 83
},
"outputId": "6386fd21-c51f-4067-c383-eb815fb2ddb6"
},
"source": [
"# A trainable variable\n",
"x0 = tf.Variable(3.0, name='x0')\n",
"# Not trainable\n",
"x1 = tf.Variable(3.0, name='x1', trainable=False)\n",
"# Not a Variable: A variable + tensor returns a tensor.\n",
"x2 = tf.Variable(2.0, name='x2') + 1.0\n",
"# Not a variable\n",
"x3 = tf.constant(3.0, name='x3')\n",
"\n",
"with tf.GradientTape() as tape:\n",
" y = (x0**5 + 3*x0**4 + 2*x0**2 + 2) + (x1**2) + (x2**2)\n",
"\n",
"grad = tape.gradient(y, [x0, x1, x2, x3])\n",
"\n",
"for g in grad:\n",
" print(g)"
],
"execution_count": 65,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(741.0, shape=(), dtype=float32)\n",
"None\n",
"None\n",
"None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "rWJuOhYRMmeP",
"colab_type": "text"
},
"source": [
"`tf.GradientTape` provides hooks that give user control over what is or is not watched.\n",
"\n",
"To record gradients with respect to a `tf.Tensor`, we need to call `GradientTape.watch(x)`:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "VeNUbJUwMwit",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "68fb667d-5f4e-457f-fe25-da7761cf658f"
},
"source": [
"x = tf.constant(3.0)\n",
"with tf.GradientTape() as tape:\n",
" tape.watch(x)\n",
" y = x**2\n",
"\n",
"# dy = 2x * dx\n",
"dy_dx = tape.gradient(y, x)\n",
"print(dy_dx.numpy())"
],
"execution_count": 66,
"outputs": [
{
"output_type": "stream",
"text": [
"6.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OooqRAdEOl2v",
"colab_type": "text"
},
"source": [
"Conversely, to disable the default behavior of watching all `tf.Variables`, set `watch_accessed_variables=False` when creating the gradient tape. This calculation uses two variables, but only connects the gradient for one of the variables:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "xUlEtEY1MlYL",
"colab_type": "code",
"colab": {}
},
"source": [
"x0 = tf.Variable(0.0)\n",
"x1 = tf.Variable(10.0)\n",
"\n",
"with tf.GradientTape(watch_accessed_variables=False) as tape:\n",
" tape.watch(x1)\n",
" y0 = tf.math.sin(x0)\n",
" y1 = tf.nn.softplus(x1)\n",
" y = y0 + y1\n",
" ys = tf.reduce_sum(y)"
],
"execution_count": 67,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "A8LLYndmNLtk",
"colab_type": "text"
},
"source": [
"Since `GradientTape.watch` was not called on `x0`, no gradient is computed with respect to it:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "AC-c8q5LNSrf",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "68cfa686-b54a-4b3b-fc8a-eb97b3412c48"
},
"source": [
"# dy = 2x * dx\n",
"grad = tape.gradient(ys, {'x0': x0, 'x1': x1})\n",
"\n",
"print('dy/dx0:', grad['x0'])\n",
"print('dy/dx1:', grad['x1'].numpy())"
],
"execution_count": 68,
"outputs": [
{
"output_type": "stream",
"text": [
"dy/dx0: None\n",
"dy/dx1: 0.9999546\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "gIcaRPwTNqM4",
"colab_type": "text"
},
"source": [
"## Intermediate Results\n",
"\n",
"We can also request gradients of the output with respect to intermediate values computed inside the `tf.GradientTape` context."
]
},
{
"cell_type": "code",
"metadata": {
"id": "_OAl_MnwNZiL",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "62e0e941-1f14-431f-a507-6ddab89c31a0"
},
"source": [
"x = tf.constant(3.0)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" tape.watch(x)\n",
" y = x * x\n",
" z = y * y\n",
"\n",
"# Use the tape to compute the gradient of z w.r.t y\n",
"# dz/dy = 2 * y, where y = x ** 2\n",
"print(tape.gradient(z, y).numpy())"
],
"execution_count": 69,
"outputs": [
{
"output_type": "stream",
"text": [
"18.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BlYvb2YxPO63",
"colab_type": "text"
},
"source": [
"The resources held by a `GradientTape` are released as soon as `GradientTape.gradient()` method is called. To compute multiple gradients over the same computation, create a `persistent` gradient tape. This allows multiple calls to the `gradient()` method as resources are released when the tape object is garbage collected."
]
},
{
"cell_type": "code",
"metadata": {
"id": "FX1IW0DSN8M1",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "a6968ebd-cc38-424c-a427-4ca46c47c5ce"
},
"source": [
"x = tf.constant([1, 3.0])\n",
"# when gradient is not persistent\n",
"with tf.GradientTape(persistent=False) as tape:\n",
" tape.watch(x)\n",
" y = x * x\n",
" z = y * y\n",
"\n",
"try:\n",
" print(tape.gradient(z, x).numpy()) # 108.0 (4 * x**3 at x = 3)\n",
" print(tape.gradient(y, x).numpy()) # 6.0 (2 * x)\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")\n"
],
"execution_count": 70,
"outputs": [
{
"output_type": "stream",
"text": [
"[ 4. 108.]\n",
"RuntimeError: GradientTape.gradient can only be called once on non-persistent tapes.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "3mu4uKBLPgVL",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "c19c2340-86ab-44d1-f29a-89b648be024d"
},
"source": [
"x = tf.constant([1, 3.0])\n",
"# when gradient is persistent\n",
"with tf.GradientTape(persistent=True) as tape:\n",
" tape.watch(x)\n",
" y = x * x\n",
" z = y * y\n",
"\n",
"try:\n",
" print(tape.gradient(z, x).numpy()) # 108.0 (4 * x**3 at x = 3)\n",
" print(tape.gradient(y, x).numpy()) # 6.0 (2 * x)\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")"
],
"execution_count": 71,
"outputs": [
{
"output_type": "stream",
"text": [
"[ 4. 108.]\n",
"[2. 6.]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "R6HlvMcVP95s",
"colab_type": "code",
"colab": {}
},
"source": [
"del tape # Drop the reference to the tape"
],
"execution_count": 72,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WcQlN74wQShH",
"colab_type": "text"
},
"source": [
"## Gradients of non-scalar targets\n",
"\n",
"A gradient is fundamentally an operation on a scalar."
]
},
{
"cell_type": "code",
"metadata": {
"id": "kEP0NQ_SQEae",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "3aadf23e-2b0b-49af-845d-248ab9456dca"
},
"source": [
"x = tf.Variable(2.0)\n",
"with tf.GradientTape(persistent=True) as tape:\n",
" y0 = x**2\n",
" y1 = 1 / x\n",
"\n",
"print(tape.gradient(y0, x).numpy())\n",
"print(tape.gradient(y1, x).numpy())"
],
"execution_count": 73,
"outputs": [
{
"output_type": "stream",
"text": [
"4.0\n",
"-0.25\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "xnEcROy-RUA3",
"colab_type": "text"
},
"source": [
"Thus, when we are asked for the gradient of multiple targets, the result for each source is:\n",
"\n",
" - The gradient of the sum of the targets, or equivalently\n",
" - The sum of the gradients of each target.\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "1Inb-6lyROD6",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "96e1379f-f67e-422f-cd9d-eb1ddf30931a"
},
"source": [
"x = tf.Variable(2.0)\n",
"with tf.GradientTape() as tape:\n",
" y0 = x**2\n",
" y1 = 1 / x\n",
"\n",
"print(tape.gradient({'y0': y0, 'y1': y1}, x).numpy())"
],
"execution_count": 74,
"outputs": [
{
"output_type": "stream",
"text": [
"3.75\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "B8U0U3lwRdH7",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "a3585a04-c9f3-49d7-933c-4b6f74fea36f"
},
"source": [
"x = tf.Variable(2.)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" y = x * [3., 4.]\n",
"\n",
"print(tape.gradient(y, x).numpy())"
],
"execution_count": 75,
"outputs": [
{
"output_type": "stream",
"text": [
"7.0\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "A53bVdyiR3fW",
"colab_type": "text"
},
"source": [
"This makes it simple to take the gradient of the sum of a collection of losses, or the gradient of the sum of an element-wise loss calculation.\n",
"\n",
"For an element-wise calculation, the gradient of the sum gives the derivative of each element with respect to its input-element, since each element is independent:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "1JWD8uEBR08t",
"colab_type": "code",
"colab": {}
},
"source": [
"x = tf.linspace(-10.0, 10.0, 200+1)"
],
"execution_count": 76,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "8j9xC7S2SIIA",
"colab_type": "code",
"colab": {}
},
"source": [
"with tf.GradientTape() as tape:\n",
" tape.watch(x)\n",
" y = tf.nn.sigmoid(x)\n",
"\n",
"dy_dx = tape.gradient(y, x)"
],
"execution_count": 77,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "5zGGoChOSO78",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 278
},
"outputId": "022eb0f2-5b75-4f86-f77c-13936060b520"
},
"source": [
"plt.plot(x, y, label='y')\n",
"plt.plot(x, dy_dx, label='dy/dx')\n",
"plt.legend()\n",
"_ = plt.xlabel('x')"
],
"execution_count": 78,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEGCAYAAAB1iW6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU9b3/8dcnk41AWANhCRhURBZFVperVEEBsaJoraCt/dW61Ft7a3d7banWent7b9t7a2ttrVZrK+LuxYoCVq1WRVkEJCCyyBIgAQKEQLZZvr8/zoBDnMAAMzmTyfv5eMwjM+d8Z+YzZybvnHznnO/XnHOIiEjrl+V3ASIikhwKdBGRDKFAFxHJEAp0EZEMoUAXEckQ2X49cVFRkSstLfXr6UVEWqXFixfvdM51j7fOt0AvLS1l0aJFfj29iEirZGYbm1unLhcRkQyhQBcRyRAKdBGRDOFbH3o8wWCQ8vJy6uvr/S6lxeXn51NSUkJOTo7fpYhIK5VWgV5eXk5hYSGlpaWYmd/ltBjnHFVVVZSXl9O/f3+/yxGRVuqIXS5m9icz225mK5pZb2Z2r5mtNbPlZjbiWIupr6+nW7dubSrMAcyMbt26tcn/TEQkeRLpQ38EmHSY9RcDA6KXm4D7j6egthbmB7TV1y0iyXPELhfn3BtmVnqYJpcBjzpvHN4FZtbZzHo557YlqUYRyVDOORpCEeqDYeqDEYLhCKGII3TwpyMU+eR6OOIIRiKEY5aHI46Ic0Qi4ICIc+DA4Yg4cM5b5qLPd/C21+zQZcSsi/48WOshdccud3GXN71P7Mrxg4oZ1rfzcW+/ppLRh94H2Bxzuzy67FOBbmY34e3F069fvyQ8tYj4xTnH7togO2oa2F5Tz679jeytD7G3LkhNfYi99cFDrtc1hg8Gd33ok+ttyYF/xHt0zE/bQE+Yc+4B4AGAUaNGaWYNkTQWjji27K5jQ9V+Nu2q9S5VtWyrrmN7TQM79zUQDMf/Nc4NZNGxXTYd83MobJdDx/xsunfIIz8nQH5OVvRn4JPb2d71nICRE8gikGXkBIxAVhbZWUZ2wKLLouuyPmmTlWUEzDCDrGhiHrh+8Cdet+ahtz+97MB9zMCIXo95XbFdo4cuj9+mpSUj0LcAfWNul0SXtTozZsyga9eu3HbbbQDccccd9OjRg2984xs+VyaSWo2hCB9s2cOKLXv5sGIvK7fV8FFFDXXB8ME2udlZ9O3Sjt6d23Fyj0J6dMyje4e8gz+7dcilY7scOubnkJ8T8PHVtF3JCPTZwK1mNgs4E6hORv/5XS+UsXLr3uMuLtbg3h358aVDml1//fXXc8UVV3DbbbcRiUSYNWsW7733XlJrEEkH9cEwizfu5t2Pd/Hex1W8v2kPDSGv+6NzQQ6DenZk2pi+DCwupLSoPSd0K6C4MJ+sLH15n86OGOhm9jhwPlBkZuXAj4EcAOfc74E5wGRgLVALfDlVxaZaaWkp3bp14/3336eyspLhw4fTrVs3v8sSSYrquiCvrKxk/spK3lizg9rGMFnm7ehce+YJjOnflTP6dqa4Y56OumqlEjnKZfoR1jvga0mrKOpwe9KpdMMNN/DII49QUVHB9ddf70sNIskSCkd4c81Onl5SzvyVlTSGIhR3zGPq8D6MO7UHo/t3pWO+zk7OFGl1pmg6mDp1KjNmzCAYDDJz5ky/yxE5JjX1QZ5cVM7Db31M+e46uhTkcM2Yflw+vA/DSjppDzxDKdCbyM3N5YILLqBz584EAvpiR1qXvfVB/vjGeh55awM1DSHGlHbljsmDGD+omNxsjcWX6RToTUQiERYsWMBTTz3ldykiCWsIhfnLOxu577W17K4NMvm0ntw89qSUHOss6UuBHmPlypV89rOfZerUqQwYMMDvckQS8u76Kn7w7Aes37mf8wYU8b2Jp3JaSSe/yxIfKNBjDB48mPXr1/tdhkhC9tYH+c+XPmTmu5vo27Udj3x5NOcP7OF3WeIjBbpIK/RBeTW3PLaYrXvquPG8/nzzolMoyNWvc1unT4BIK+Kc4/H3NnPn7DKKOuTy1FfPYeQJXfwuS9KEAl2klQiFI/zw+RXMWriZ8wYU8etpw+naPtfvsiSNKNBFWoG6xjBff3wJr6zazq0XnMw3LzqFgE7DlyZ0YOoR3HnnnfziF784bJtZs2Zxzz33fGp5aWkpO3fuTFVp0kZU1wb54kPv8vcPt3P35UP5zsSBCnOJS4GeBC+99BKTJh1uUieRY7O3Psg1Dy5gWfkefjt9BF886wS/S5I0pkCP45577uGUU07h3HPPZfXq1YTDYUaM+GSq1DVr1hy87Zxj6dKljBgxgqqqKiZMmMCQIUO44YYbcNEZShYuXMjpp59OfX09+/fvZ8iQIaxYEXeKVpGD6hrD3PDIIlZX1PDAdaO45PRefpckaS59+9Bfuh0qPkjuY/Y8DS7+z8M2Wbx4MbNmzWLp0qWEQiFGjBjByJEj6dSpE0uXLuWMM87g4Ycf5stf9gaVfP/99xk2bBhmxl133cW5557LjBkzePHFF3nooYcAGD16NFOmTOGHP/whdXV1fOELX2Do0KHJfW2SUYLhCF+buYSFG3dx77ThXKDjyyUB6RvoPnnzzTeZOnUqBQUFAEyZMgXwRmF8+OGH+dWvfsUTTzxxcJz0l19+mYsvvhiAN954g2effRaASy65hC5dPjmcbMaMGYwePZr8/HzuvffelnxJ0so45/j+M8t59cPt3DN1KJcO6+13SdJKpG+gH2FPuqVdeeWV3HXXXYwbN46RI0ceHCd93rx5PPPMM0e8f1VVFfv27SMYDFJfX0/79u1TXbK0Un96awPPLtnCNy88hWvPVJ+5JE596E2MHTuW559/nrq6OmpqanjhhRcAyM/PZ+LEidxyyy0Hu1uqq6sJhUIHw33s2LEHh9x96aWX2L1798HHvfnmm7n77ru59tpr+f73v9/Cr0pai3fWVfEfc1YxcUgx/zb+ZL/LkVYmfffQfTJixAiuvvpqhg0bRo8ePRg9evTBdddeey3PPfccEyZMAGD+/PlceOGFB9f/+Mc/Zvr06QwZMoRzzjmHfv36AfDoo4+Sk5PDNddcQzgc5pxzzuHVV19l3LhxLfviJK1tq67j1plLKO1WwC+uGqYxy+Wo2YEjMVraqFGj3KJFiw5ZtmrVKgYNGuRLPYn4xS9+QXV1NXfffTfg9avfcMMNnHXWWUl5/HR//ZI6oXCEq/7wDmsq9/H81/6Fk3t08LskSVNmttg5NyreOu2hJ2jq1KmsW7eOV1999eCyBx980MeKJJP84Y31vL9pD/dOH64wl2OmQE/Qc88953cJkqFWbt3L/77yEZec3ospOqJFjkPafSnqVxeQ39rq627rGkMRvv3UMjq1y+Xuy3RughyftAr0/Px8qqqq2ly4OeeoqqoiPz/f71Kkhf3m1TWs2raXn11xmkZOlOOWVl0uJSUllJeXs2PHDr9LaXH5+fmUlJT4XYa0oLXba7j/9XVcMaIPFw0u9rscyQBpFeg5OTn079/f7zJEUs45x10vrKQgN8Adk3VkkyRHWnW5iLQVc8sqeXPNTr510Sl065DndzmSIRToIi2sPhjmpy+uZGBxIV/QcLiSRGnV5SLSFvzhH+sp313H4zeeRXZA+1SSPPo0ibSg7TX13P+PtVxyWi/OPqmb3+VIhlGgi7Sg3722jmDY8d2JA/0uRTKQAl2khWzdU8fMdzdx1cgSSos0fLIknwJdpIX85tU1AHx9/ACfK5FMlVCgm9kkM1ttZmvN7PY46/uZ2Wtm9r6ZLTezyckvVaT12rBzP08uKmf6mL706dzO73IkQx0x0M0sANwHXAwMBqab2eAmzX4IPOmcGw5MA36X7EJFWrN7/76G7Czjaxdo0gpJnUT20McAa51z651zjcAs4LImbRzQMXq9E7A1eSWKtG6bqmp5fukWrjv7BHp01Hg9kjqJBHofYHPM7fLoslh3Al8ws3JgDvD1eA9kZjeZ2SIzW9QWx2uRtunBf64nkGXccN6JfpciGS5ZX4pOBx5xzpUAk4G/mNmnHts594BzbpRzblT37t2T9NQi6WvX/kaeXLSZy8/oQ7H2ziXFEgn0LUDfmNsl0WWxvgI8CeCcewfIB4qSUaBIa/boOxuoD0a4aaz2ziX1Egn0hcAAM+tvZrl4X3rObtJmEzAewMwG4QW6+lSkTatrDPPoOxsZd2oPBhQX+l2OtAFHDHTnXAi4FZgLrMI7mqXMzH5iZlOizb4N3Ghmy4DHgf/n2tosFSJNPL2knF37G7V3Li0mocG5nHNz8L7sjF02I+b6SuBfkluaSOsVjjgeenM9w0o6cWb/rn6XI22EzhQVSYE3PtrBhqpavnLeiZiZ3+VIG6FAF0mBvy7YSFGHPCYN6el3KdKGKNBFkqx8dy2vrt7O1aNLyM3Wr5i0HH3aRJLs8fc2YcD0Mf38LkXaGAW6SBI1hiI8sXAz407tQUmXAr/LkTZGgS6SRHPLKti5r5FrNVeo+ECBLpJEf12wkb5d2/GZARraQlqeAl0kSdbv2Me7H+/imjEnkJWlQxWl5SnQRZLkmSXlBLKMK0c0HYxUpGUo0EWSIBxxPLtkC2MHFGnMc/GNAl0kCd5et5Nt1fV8bmTfIzcWSREFukgSPL24nE7tchg/qIffpUgbpkAXOU5764O8vKKCKcN6k58T8LscacMU6CLH6cXl22gIRfjcyBK/S5E2ToEucpyeXlzOgB4dOL2kk9+lSBunQBc5Dht27mfxxt1cObJEw+SK7xToIsfhhWVbAZgyrLfPlYgo0EWOmXOO2cu2Mqa0K707t/O7HBEFusix+rCihjXb93HpGdo7l/SgQBc5RrOXbSWQZUweqlmJJD0o0EWOgXOOF5Zt5dyTi+jWIc/vckQABbrIMVmyaQ/lu+v0ZaikFQW6yDF4YdlW8rKzmDCk2O9SRA5SoIscpVA4wt+Wb2PcqT0ozM/xuxyRgxToIkdp4Ybd7NzXwKXqbpE0o0AXOUpzyyrIy87i/IGaZk7SiwJd5Cg455i/spLzBnSnIDfb73JEDqFAFzkKZVv3smVPnb4MlbSkQBc5CvPKKsgyuHCQAl3SjwJd5CjMLatkdGlXurbP9bsUkU9JKNDNbJKZrTaztWZ2ezNtPm9mK82szMxmJrdMEf9t2Lmf1ZU1TByiU/0lPR3xWx0zCwD3ARcB5cBCM5vtnFsZ02YA8APgX5xzu81MEytKxpm3sgJA/eeSthLZQx8DrHXOrXfONQKzgMuatLkRuM85txvAObc9uWWK+G9uWSVDenekpEuB36WIxJVIoPcBNsfcLo8ui3UKcIqZvWVmC8xsUrwHMrObzGyRmS3asWPHsVUs4oPtNfUs2bRb3S2S1pL1pWg2MAA4H5gO/NHMOjdt5Jx7wDk3yjk3qnt3nZQhrccrK7fjnLpbJL0lEuhbgL4xt0uiy2KVA7Odc0Hn3MfAR3gBL5IR5pZVcEK3AgYWF/pdikizEgn0hcAAM+tvZrnANGB2kzbP4+2dY2ZFeF0w65NYp4hvauqDvL1uJxMGF2siaElrRwx051wIuBWYC6wCnnTOlZnZT8xsSrTZXKDKzFYCrwHfdc5VpapokZb02uodBMNO/eeS9hIajMI5NweY02TZjJjrDvhW9CKSUeaVVVDUIY/h/br4XYrIYelMUZHDaAiFeX31Di4a3INAlrpbJL0p0EUO4+11VexrCDFB3S3SCijQRQ5jXlkFHfKyOeekbn6XInJECnSRZoQj3tjn5w/sTl52wO9yRI5IgS7SjPc37WbnvkZ1t0iroUAXacbcsgpyA1lcoKnmpJVQoIvE4Zxj3spKzjm5G4X5OX6XI5IQBbpIHKsra9hYVcuEwepukdZDgS4Sx9wVlZjBhYM1tL+0Hgp0kTjmraxgRL8u9CjM97sUkYQp0EWa2LyrlrKte5mooXKllVGgizQxf2UlgPrPpdVRoIs0MbesgoHFhZQWtfe7FJGjokAXiVG1r4GFG3apu0VaJQW6SIy/f7idiENnh0qrpEAXiTGvrII+ndsxpHdHv0sROWoKdJGo/Q0h3lizk4s01Zy0Ugp0kag3PtpBYyiiqeak1VKgi0TNLaugS0EOo0s11Zy0Tgp0ESAYjvD3D7czflAx2QH9WkjrpE+uCLBgfRU19SF1t0irpkAXAeaVVdIuJ8B5A4r8LkXkmCnQpc2LRBzzVlbwmVO6k5+jqeak9VKgS5u3fEs1lXsbmKCzQ6WVU6BLmze3rIJAljH+VAW6tG4KdGnz5pZVcNaJXelUoKnmpHVToEubtnb7Ptbv2K+jWyQjKNClTZtbVgHARYPV3SKtnwJd2rR5ZRUMK+lEr07t/C5F5Lgp0KXN2rKnjmXl1Uwcqu4WyQwKdGmzXl7hdbdcPLSXz5WIJEdCgW5mk8xstZmtNbPbD9PuSjNzZjYqeSWKpMbLK7Zxas9C+muqOckQRwx0MwsA9wEXA4OB6WY2OE67QuAbwLvJLlIk2bbX1LNo424mqbtFMkgie+hjgLXOufXOuUZgFnBZnHZ3Az8H6pNYn0hKzC2rxDl1t0hmSSTQ+wCbY26XR5cdZGYjgL7OuRcP90BmdpOZLTKzRTt27DjqYkWS5eUV2zixqD2nFHfwuxSRpDnuL0XNLAv4FfDtI7V1zj3gnBvlnBvVvXv3431qkWOye38jC9bvYtLQnppqTjJKIoG+Begbc7skuuyAQmAo8LqZbQDOAmbri1FJV/NXVhKOOHW3SMZJJNAXAgPMrL+Z5QLTgNkHVjrnqp1zRc65UudcKbAAmOKcW5SSikWO00srtlHSpR1D+3T0uxSRpDpioDvnQsCtwFxgFfCkc67MzH5iZlNSXaBIMu2tD/LPtTuZNETdLZJ5shNp5JybA8xpsmxGM23PP/6yRFLj1VXbCYYdF5+mwxUl8+hMUWlTXlqxjeKOeQzv28XvUkSSToEubUZtY4h/fLSDiUN6kpWl7hbJPAp0aTNeX72D+mBEZ4dKxlKgS5vxwrKtFHXIZUxpV79LEUkJBbq0CTX1Qf7+4XYuOa0X2QF97CUz6ZMtbcK8skoaQxGmnNHb71JEUkaBLm3C7GVb6dO5HSP66egWyVwKdMl4Vfsa+OfanVw6rLdOJpKMpkCXjDdnRQXhiGPKMHW3SGZToEvGe2HZVk7u0YFBvQr9LkUkpRToktG2VdexcMMuLj1d3S2S+RToktFmL92Kc+joFmkTFOiSsZxzPLOknOH9OmsiaGkTFOiSsT7YUs1Hlfv43MgSv0sRaREKdMlYTy8uJzc7i8+eru4WaRsU6JKRGkJhZi/bysQhPenULsfvckRahAJdMtKrq7azpzao7hZpUxTokpGeXlxOccc8zj25yO9SRFqMAl0yzvaael7/aAdXjCghoIkspA1RoEvGeW7JFsIRx5Uj1N0ibYsCXTJKJOKY+d4mRpd24eQeHfwuR6RFKdAlo/xz7U42VtXyhbNO8LsUkRaX7XcBIsn01wUb6dY+99jmDW2ogT2boH4v5HeCLidArs4wldZDgS4ZY1t1Ha+squSmsSeRlx1I7E61u2DpTFjxNGxbBi7yybqsbOg9HE67Ck6/Gtp1Tk3hIkmiQJeM8fi7m3DAtWf2O3LjYB28dS+8fS807oM+I2Hsd6HHIMjrCPV7oHIlrJ0PL30PXr0HzvsWnHULZOel/LWIHAsFumSEYDjCrIWbOf+U7vTtWnD4xuWL4bmboWoNDLoUzv8BFA/5dLuhV8L4H8HWpfD6z+CVH8MHT8HU30PP01LzQkSOg74UlYwwf2Ul22sajvxl6KKH4U8TIVgLX3wOrv5r/DCP1fsMuOYJmP4E7NsOD14Iy59MXvEiSaJAl4zw0D8/pqRLO84f2CN+g0gE5t4Bf7sNTvwM3PIWnDTu6J5k4CS45W3oMwqevRH+8d/HX7hIEinQpdVbvHEXizfu5ivn9o9/ZmgkDLNvhXd+C2NugmuehHZdju3JOnSH656H06fBaz+Fv/8EnDu+FyCSJOpDl1bvD/9YT6d2OXx+VN9Pr3QO/vZNWPqY11f+me/D8U5FF8iBy+/3vhx985cQaoAJPz3+xxU5TgntoZvZJDNbbWZrzez2OOu/ZWYrzWy5mf3dzHRWh7SI9Tv2MX9VJdedfQLt85rsnzgH834IS/4M530bzr89eaGblQWX/hrG3Ozt+b9yZ3IeV+Q4HHEP3cwCwH3ARUA5sNDMZjvnVsY0ex8Y5ZyrNbNbgP8Crk5FwSKx/vjmx+QEsrju7NJPr/zHz72wPfOrMO5HyX9yM7j45xAJwVv/Cx17w5k3J/95RBKUyB76GGCtc269c64RmAVcFtvAOfeac642enMBoFGRJOV21DTwzJJyrhxRQvfCJseGL3vCO9TwjGth4s9S1x1iBpP/GwZeAi99H1b+X2qeRyQBiQR6H2BzzO3y6LLmfAV4Kd4KM7vJzBaZ2aIdO3YkXqVIHI+8/THBcIQbz+t/6IryxTD761B6ntctkpXi7/6zAnDlg1AyGp65ETYtSO3ziTQjqZ90M/sCMAqIezyXc+4B59wo59yo7t27J/OppY2p2tfAI29tYPLQXpzYPWZUxb3bYNY1UNgTrvqz9wVmS8gt8I5V79QHnvgC7Nl85PuIJFkigb4FiD18oCS67BBmdiFwBzDFOdeQnPJE4vvDG+upC4b55kUDPlkYrPPCvHEfTJ8F7bu1bFEFXb3nDdZH66g98n1EkiiRQF8IDDCz/maWC0wDZsc2MLPhwB/wwnx78ssU+cT2vfX8+e0NXD68Dyf3KPQWOgcvfAO2LoErHoDiwf4U132g1/1S8YF37LuOUZcWdMRAd86FgFuBucAq4EnnXJmZ/cTMpkSb/TfQAXjKzJaa2exmHk7kuN332lrCEcdt40/5ZOHb98LyJ+CCH8Kpl/hXHHhnlI7/Eax4Bv75P/7WIm1KQicWOefmAHOaLJsRc/3CJNclElf57lpmvreJz4/uS79u0UG4PpoH838Mgy+Hsd/xt8ADzv0WVJZ5Z5L2GOyFvEiK6dR/aVX+Z/4azIyvjzvZW7DjI3jmK97oh5f/Ln3O1jSDKb+FXqfDMzfAjtV+VyRtgAJdWo0lm3bzzJJyvvwvpfTq1A7qdsPj07xT8KfNTL/ZhXILvLpy8r0663b7XZFkOAW6tAqRiOPO2WX0KMzj6+MGQDgET33ZmzLu6r9C5zjjuKSDTiVefXs2w9PXe3WLpIgCXVqFpxZvZnl5Nf8+eRAd8rJh/gxY/xp89n+g31l+l3d4/c6CS34J6171JskQSRGNtihpr7ouyH+9vJrRpV247IzesPjPsOA+b4yWEV/0u7zEjPwSVK7wxpYpHgpnTPe7IslA2kOXtPfLeavZXdvInVOGYBvehBe/BSeNhwn3+F3a0Zn4H9B/LLzwb7B5od/VSAZSoEtae3vdTh59ZyPXnV3KkLyd8MQXodvJcNXDEGhl/2AGcrzhCDr2hieuhb1b/a5IMowCXdJWTX2Q7z61nP5F7fn+Z4ph5ue9gbCmz4L8Tn6Xd2wKusK0x6FxP8y6VsMDSFIp0CVt3fPiKrZV1/GLKwfT7vkDR7Q8Bl37H/nO6ax4sDc8wdb3vblJI2G/K5IMoUCXtPTah9uZtXAzN489kZHL74KP34BL74UTzva7tOQ49RJvcowP/+aNo64xXyQJWlknpLQFW/bU8e2nljGwuJBvB2bB+3+Fsd/LvCNDzrwZqsu9cWg6lcC5t/ldkbRyCnRJK/XBMLf8dTHBUITHhrxH9tv/CyO/DBf8u9+lpcaFd8HeLd7x6e26eIc3ihwjBbqkDeccP3p+BcvLq/nbuRsoevtub8CtS36ZPmO0JFtWFlx+P9RXe8P/5rSD0z/vd1XSSqkPXdLGX9/dxFOLy/n9kFUMXXQHnHiB9+VhVsDv0lIrO88bHqD0XHjuq5qXVI6ZAl3SwssrKvjx/61gRu9FTFz3UzjpApj+uBd2bUFOO+9wzJJR3pgvZc/5XZG0Qgp08d0/1+zk3x5/n+8VvcX1u36FnTzeO1Y7p53fpbWsvA5w7VPQJxrqi//sd0XSyijQxVfvb9rNTX9ZyF0dnuWrNffBgIneseY5+X6X5o/8TvDF5+Ckcd4QAW/92u+KpBVRoItvFqyv4vqH3uZXOb9nesOTMPyLMK0Nh/kBuQXefyiDL/dGlXzxOxAO+l2VtAI6ykV88fKKCu6c9ToP5/2WM8IfwAV3wNjvZu7RLEcrOxc+9yeYX+KN0LjzI7jqEW/oAJFmaA9dWtxj727koZkzeTH33xlma2DqH+Az31OYN5UVgIn3wGW/g03vwIPjoeIDv6uSNKZAlxZTHwxz+1NLWT/75zye+1O6dOqIfWU+DJvmd2npbfi18KW/eQN5/XEcLLhfQwVIXAp0aREbq/bzr795hqkf3MyPch4jMHAiWTe97k2iLEfW70y45S1vHPiXb/dGnty7ze+qJM2oD11SKhxxPPb2Osrn/Yb7sh4nJy8HJt+HnXGtuliOVvsi79j8hQ/CvB/CfWNg/AwYdX3mn3wlCVGgS8p8VFnDzMf/zPRd93NdVjn1J1xA9hW/9QaikmNjBmNu9A5rfPHbMOc7sHQmTPpPby9e2jQFuiRd5d56npz9AoM/+h13Zi1hf4e+uM/+hfxBl2qvPFm6neQdr77iGZj77/CnCTBwMoz7kTfeurRJ5nz6cmXUqFFu0aJFvjy3pMb26jpenjubfmX3c769T12gkMg536D92K/r2PJUatzvfVH61q+hoQYGXQrnfB36jvG7MkkBM1vsnBsVd50CXY7Xyk0VfPDSQwzZ+hRD7WP2BToSGvM1On/mXyG/o9/ltR21u+Dt38Cih7zRG0vGeN0zgy5te8MoZDAFuiTdzr37WfTa8wTKnuXMhrfoaHVU5p9I9lk30u3s67xxScQfDftg6WOw4HewewPkdYKhV8Bpn4N+Z+sL1FZOgS7HzTnHhvItrF/wAtnrX2FI7XsU2V72WwHbeo6n5/k30uGUseojTyeRCGx8y5vxaeX/QagOCopg4MUwYII3XK/OPG11FOhy1CIRx8frVrP1g9dwG9+hZ0rE928AAAsoSURBVPUyTnIbCZhjrxWypds5dBp5Jb1HXab+8dagYR+sfQVWvQAfzYXGGsCgeCj0P88L915nQMfe+qOc5hTo0iznHDt3VLJ94yqqNy4jUrGSwurV9Gn8mCKrBmA/+WwuGEJjr9EUj5hM8aBz9W97axYOwpYl3sTbG96Aze9BqN5bV9ANep4OvYZBj0HQ7WToeqL25NPIcQe6mU0Cfg0EgAedc//ZZH0e8CgwEqgCrnbObTjcYyrQUy8cDrN7ZwXVO8rZt3ML9Xu2Ea6uwPZXkrNvK53qt1AcrqCj1R68Tx25bM05gb0dT4Few+g19HyKB4zAAjk+vhJJqVADbF0KFcth2zLvsn0VRGJGeGzXBbqe5O3BF/aCjr2gsDcU9vRuF3SF/M4Q0JHQqXa4QD/i1jezAHAfcBFQDiw0s9nOuZUxzb4C7HbOnWxm04CfA1cff+mZIRIOEwoFiYRDhEJBwqEQ4VAjkVCIUDhIJBQiHA4SCYeJhIOEg42EGusINdQSbqwn3FhLpLHOuwTrIViHC9VDsJ6sUC1ZjXvJDtaQF9pHXmgfBZF9tHe1dKCWInMUNamn1uVRFShiT14fPuwwHOtSSn7xSRSfdAbd+w7kJP1Sti3Zed5JSbEnJoUavS9Uq9bCrnVQtQ52rYcdH8L616Fhb/zHyi2Edp29cG8XveR18o6yyS2AnALv+sGf0evZ+RDIgawc749CVo53O5ALWdnx11lWzEXdRJDYiUVjgLXOufUAZjYLuAyIDfTLgDuj158Gfmtm5lLQn7Pw2V/TY8UDABgOi3kKwwHOW35wqdfmkNvRS+z94t1u7j4W87gHbsd7jABhsomQZY7cJLz2phpdgHrLY7+1pzarAw2BDtTk92JXTiGR3EJcXkesfRG5nXtT0LUXhUUldOlRQkGHThSY0TcFNUmGyM6F7qd4l3ga9kHNtuilAup2Q90eqN9z6PWda73wD9ZBsPaTrp1UiA14rEngR0M/7vUD7Q/8xtshPw5d1qRNossO+YNjcP73YeiVx/Vy40kk0PsAm2NulwNNzzE+2MY5FzKzaqAbsDO2kZndBNwE0K9fv2MqOKewO1UFJ30St/ZJnH5yGw7GbnR97BsWt601fYzYNyfrkHYHHyO2XZz7uKxsLCsblxXw+pyzcrCsAGRlY4Fs72dWNmQFyApkQyCbrKxsLJBDICefQF4B2bn55OS3Jye/gNz8duTltSevXXty8wvIzc4mF9CR3tLi8jpA3gAoGnB094tEvKNtDgR8sM47MSpU7/XtR4IQDkV/BmOWBSESOvQ2zht10kWaubgmP5tccIe2hZhRLGP2RZsuO2Q/NZFlcR4rv/PRbbcEtej/1s65B4AHwOtDP5bHOOOia+Cia5Jal4i0kKwsyG3vXSTpEhk+dwsc8t95SXRZ3DZmlg10wvtyVEREWkgigb4QGGBm/c0sF5gGzG7SZjbwpej1zwGvpqL/XEREmnfELpdon/itwFy8wxb/5JwrM7OfAIucc7OBh4C/mNlaYBde6IuISAtKqA/dOTcHmNNk2YyY6/XAVcktTUREjoamoBMRyRAKdBGRDKFAFxHJEAp0EZEM4dtoi2a2A9h4jHcvoslZqGlCdR0d1XX00rU21XV0jqeuE5xz3eOt8C3Qj4eZLWputDE/qa6jo7qOXrrWprqOTqrqUpeLiEiGUKCLiGSI1hroD/hdQDNU19FRXUcvXWtTXUcnJXW1yj50ERH5tNa6hy4iIk0o0EVEMkTaBrqZXWVmZWYWMbNRTdb9wMzWmtlqM5vYzP37m9m70XZPRIf+TXaNT5jZ0uhlg5ktbabdBjP7INou5TNjm9mdZrYlprbJzbSbFN2Ga83s9hao67/N7EMzW25mz5lZ3GlbWmp7Hen1m1le9D1eG/0slaaqlpjn7Gtmr5nZyujn/xtx2pxvZtUx7++MeI+VgtoO+76Y597o9lpuZiNaoKaBMdthqZntNbPbmrRpse1lZn8ys+1mtiJmWVczm29ma6I/uzRz3y9F26wxsy/Fa3NEzrm0vACDgIHA68ComOWDgWVAHtAfWAcE4tz/SWBa9PrvgVtSXO8vgRnNrNsAFLXgtrsT+M4R2gSi2+5EIDe6TQenuK4JQHb0+s+Bn/u1vRJ5/cC/Ar+PXp8GPNEC710vYET0eiHwUZy6zgf+1lKfp0TfF2Ay8BLeXIxnAe+2cH0BoALvxBtfthcwFhgBrIhZ9l/A7dHrt8f73ANdgfXRn12i17sc7fOn7R66c26Vc251nFWXAbOccw3OuY+BtXgTWR9kZgaMw5uwGuDPwOWpqjX6fJ8HHk/Vc6TAwcm/nXONwIHJv1PGOTfPOReK3lyAN/uVXxJ5/ZfhfXbA+yyNj77XKeOc2+acWxK9XgOswpuztzW4DHjUeRYAnc2sVws+/3hgnXPuWM9AP27OuTfw5oSIFfs5ai6LJgLznXO7nHO7gfnApKN9/rQN9MOIN2l10w98N2BPTHjEa5NM5wGVzrk1zax3wDwzWxydKLsl3Br9t/dPzfyLl8h2TKXr8fbm4mmJ7ZXI6z9k8nPgwOTnLSLaxTMceDfO6rPNbJmZvWRmQ1qopCO9L35/pqbR/E6VH9vrgGLn3Lbo9QqgOE6bpGy7Fp0kuikzewXoGWfVHc65/2vpeuJJsMbpHH7v/Fzn3BYz6wHMN7MPo3/JU1IXcD9wN94v4N143UHXH8/zJaOuA9vLzO4AQsBjzTxM0rdXa2NmHYBngNucc3ubrF6C162wL/r9yPPAgBYoK23fl+h3ZFOAH8RZ7df2+hTnnDOzlB0r7mugO+cuPIa7JTJpdRXev3vZ0T2reG2SUqN5k2JfAYw8zGNsif7cbmbP4f27f1y/CIluOzP7I/C3OKsS2Y5Jr8vM/h/wWWC8i3YexnmMpG+vOI5m8vNya8HJz80sBy/MH3POPdt0fWzAO+fmmNnvzKzIOZfSQagSeF9S8plK0MXAEudcZdMVfm2vGJVm1ss5ty3aBbU9TpsteH39B5TgfX94VFpjl8tsYFr0CIT+eH9p34ttEA2K1/AmrAZvAutU7fFfCHzonCuPt9LM2ptZ4YHreF8MrojXNlma9FtObeb5Epn8O9l1TQK+B0xxztU206altldaTn4e7aN/CFjlnPtVM216HujLN7MxeL/HKf1Dk+D7Mhu4Lnq0y1lAdUxXQ6o1+1+yH9uridjPUXNZNBeYYGZdol2kE6LLjk5LfPN7LBe8ICoHGoBKYG7MujvwjlBYDVwcs3wO0Dt6/US8oF8LPAXkpajOR4CvNlnWG5gTU8ey6KUMr+sh1dvuL8AHwPLoh6lX07qityfjHUWxroXqWovXT7g0evl907pacnvFe/3AT/D+4ADkRz87a6OfpRNbYBudi9dVtjxmO00GvnrgcwbcGt02y/C+XD6nBeqK+740qcuA+6Lb8wNijk5LcW3t8QK6U8wyX7YX3h+VbUAwml9fwfve5e/AGuAVoGu07SjgwZj7Xh/9rK0Fvnwsz69T/0VEMkRr7HIREZE4FOgiIhlCgS4ikiEU6CIiGUKBLiKSIRToIiIZQoEuIpIhFOgiUWY2OjqgWX70zMgyMxvqd10iidKJRSIxzOyneGeItgPKnXM/87kkkYQp0EViRMd1WQjU450iHva5JJGEqctF5FDdgA54swXl+1yLyFHRHrpIDDObjTd7UX+8Qc1u9bkkkYT5Oh66SDoxs+uAoHNuppkFgLfNbJxz7lW/axNJhPbQRUQyhPrQRUQyhAJdRCRDKNBFRDKEAl1EJEMo0EVEMoQCXUQkQyjQRUQyxP8H/f40iDElTOEAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "u4e4OiMpTMCX",
"colab_type": "text"
},
"source": [
"## Control flow\n",
"\n",
"Because tapes record operations as they are executed, Python control flow (using `if`s and `while`s for example) is naturally handled.\n",
"\n",
"Here a different variable is used on each branch of an `if`. The gradient only connects to the variable that was used:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "7HkKAo-6SRyP",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "4ceed639-2c3c-4e81-81fd-d0741f81fc1f"
},
"source": [
"x = tf.constant(1.0)\n",
"\n",
"v0 = tf.Variable(2.0)\n",
"v1 = tf.Variable(2.0)\n",
"\n",
"with tf.GradientTape(persistent=True) as tape:\n",
" tape.watch(x)\n",
" if x > 0.0:\n",
" result = v0\n",
" else:\n",
" result = v1**2 \n",
"\n",
"dv0, dv1 = tape.gradient(result, [v0, v1])\n",
"\n",
"print(dv0)\n",
"print(dv1)"
],
"execution_count": 79,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor(1.0, shape=(), dtype=float32)\n",
"None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "JALvw4ntThPs",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "16758cc6-7e3c-4b3c-a7fa-621d96d9d012"
},
"source": [
"dx = tape.gradient(result, x)\n",
"print(dx)"
],
"execution_count": 80,
"outputs": [
{
"output_type": "stream",
"text": [
"None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "acV5Bk_zVPvi",
"colab_type": "text"
},
"source": [
"## Getting a gradient of `None`\n",
"\n",
"When a target is not connected to a source you will get a gradient of `None`."
]
},
{
"cell_type": "code",
"metadata": {
"id": "jEwhzyZKVLH9",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "01402729-8387-4645-cd5c-de35ae893018"
},
"source": [
"x = tf.Variable(2.)\n",
"y = tf.Variable(3.)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" z = y * y\n",
" \n",
"print(tape.gradient(z, x))"
],
"execution_count": 81,
"outputs": [
{
"output_type": "stream",
"text": [
"None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5P44uHqDVxOg",
"colab_type": "text"
},
"source": [
"### Replaced a variable with a tensor \n",
"\n",
"In the section on \"Controlling what the tape watches\" we saw that the tape will automatically watch a `tf.Variable` but not a `tf.Tensor`.\n",
"\n",
"One common error is to inadvertently replace a `tf.Variable` with a `tf.Tensor`, instead of using `Variable.assign` to update the `tf.Variable`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "XMlyEeQAVaIZ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "e603167f-8f4d-49cc-f826-6a1c5145f607"
},
"source": [
"x = tf.Variable(2.0)\n",
"\n",
"for epoch in range(2):\n",
" with tf.GradientTape() as tape:\n",
" y = x+1\n",
"\n",
" print(type(x).__name__, \":\", tape.gradient(y, x))\n",
" x = x + 1 # This should be `x.assign_add(1)`"
],
"execution_count": 82,
"outputs": [
{
"output_type": "stream",
"text": [
"ResourceVariable : tf.Tensor(1.0, shape=(), dtype=float32)\n",
"EagerTensor : None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Bq4hw7hnWSQW",
"colab_type": "text"
},
"source": [
"### Did calculations outside of TensorFlow\n",
"\n",
"The tape can't record the gradient path if the calculation exits TensorFlow. For example:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "b9y8xkg_WDjZ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "8c7e6ec0-6e4e-4ae2-a514-bbfbfa4ac26c"
},
"source": [
"x = tf.Variable([[1.0, 2.0],\n",
" [3.0, 4.0]], dtype=tf.float32)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" x2 = x**2\n",
"\n",
" # This step is calculated with NumPy\n",
" y = np.mean(x2, axis=0)\n",
"\n",
" # Like most ops, reduce_mean will cast the NumPy array to a constant tensor\n",
" # using `tf.convert_to_tensor`.\n",
" y = tf.reduce_mean(y, axis=0)\n",
"\n",
"print(tape.gradient(y, x))"
],
"execution_count": 83,
"outputs": [
{
"output_type": "stream",
"text": [
"None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2z_AEk4lYqq3",
"colab_type": "text"
},
"source": [
"### Took gradients through an integer or string\n",
"\n",
"Integers and strings are not differentiable. If a calculation path uses these data types there will be no gradient.\n",
"\n",
"Nobody expects strings to be differentiable, but it's easy to accidentally create an int constant or variable if we don't specify the `dtype`."
]
},
{
"cell_type": "code",
"metadata": {
"id": "RMdLb8zSX9Vn",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 86
},
"outputId": "757d91a1-36f0-4d26-faa6-05aa3906866d"
},
"source": [
"x = tf.Variable([[2, 2],\n",
" [2, 2]])\n",
"\n",
"with tf.GradientTape() as tape:\n",
" # The path to x1 is blocked by the `int` dtype here.\n",
" y = tf.cast(x, tf.float32)\n",
" y = tf.reduce_sum(x)\n",
"\n",
"\n",
"try:\n",
" print(tape.gradient(y, x))\n",
"except Exception as e:\n",
" print(f\"{type(e).__name__}: {e}\")"
],
"execution_count": 84,
"outputs": [
{
"output_type": "stream",
"text": [
"WARNING:tensorflow:The dtype of the target tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n",
"WARNING:tensorflow:The dtype of the source tensor must be floating (e.g. tf.float32) when calling GradientTape.gradient, got tf.int32\n",
"None\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OxiIEbaCZsCT",
"colab_type": "text"
},
"source": [
"TensorFlow doesn't automatically cast between types, so in practice you'll often get a type error instead of a missing gradient."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ArRuVlm-Z3bP",
"colab_type": "text"
},
"source": [
"## No gradient registered\n",
"\n",
"Some `tf.Operations` are **registered as being non-differentiable** and will return `None`. Others have **no gradient registered**.\n",
"\n",
"The `tf.raw_ops` page shows which low-level ops have gradients registered.\n",
"\n",
"If you attempt to take a gradient through a float op that has no gradient registered the tape will throw an error instead of silently returning None. This way you know something has gone wrong.\n",
"\n",
"For example the `tf.image.adjust_contrast` function wraps `raw_ops.AdjustContrastv2` which could have a gradient but the gradient is not implemented:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "yNvyzXpoZXeC",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "3a98e6f8-fd8b-4134-d5ba-e0bf96142050"
},
"source": [
"image = tf.Variable([[[0.5, 0.0, 0.0]]])\n",
"delta = tf.Variable(0.1)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" new_image = tf.image.adjust_contrast(image, delta)\n",
"\n",
"try:\n",
" print(tape.gradient(new_image, [image, delta]))\n",
" assert False # This should not happen.\n",
"except LookupError as e:\n",
" print(f'{type(e).__name__}: {e}')"
],
"execution_count": 85,
"outputs": [
{
"output_type": "stream",
"text": [
"LookupError: gradient registry has no entry for: AdjustContrastv2\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FQCcxIroa6yf",
"colab_type": "text"
},
"source": [
"If we need to differentiate through this op we can implement the gradient and register it (using `tf.RegisterGradient`), or re-implement the function using other ops."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "opIwCyfGbpAi",
"colab_type": "text"
},
"source": [
"## Zeros instead of `None`\n",
"In some cases it would be convenient to get 0 instead of None for unconnected gradients. We can decide what to return when we have unconnected gradients using the `unconnected_gradients` argument:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "yRbOhiY7anh9",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "6de782ce-3836-4ae4-947a-43897c0a4df8"
},
"source": [
"x = tf.Variable([2., 2.])\n",
"y = tf.Variable(3.)\n",
"\n",
"with tf.GradientTape() as tape:\n",
" z = y**2\n",
"print(tape.gradient(z, x, unconnected_gradients=tf.UnconnectedGradients.ZERO))"
],
"execution_count": 86,
"outputs": [
{
"output_type": "stream",
"text": [
"tf.Tensor([0. 0.], shape=(2,), dtype=float32)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "uzRlblJddKmh",
"colab_type": "text"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WAfMbcS3nIk6",
"colab_type": "text"
},
"source": [
"# <h1 align=\"center\">-----Part - 3-----</h1>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "nn7d2B4em3dD",
"colab_type": "text"
},
"source": [
"## Logistic Regression with Tensorflow"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "M4PRDo7Sow-1",
"colab_type": "text"
},
"source": [
"### Import packages"
]
},
{
"cell_type": "code",
"metadata": {
"id": "EQ9xnnCynDev",
"colab_type": "code",
"colab": {}
},
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"from IPython.display import clear_output\n",
"import urllib\n",
"\n",
"# hide warnings\n",
"import warnings\n",
"warnings.filterwarnings('ignore')\n",
"\n",
"import tensorflow as tf"
],
"execution_count": 87,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "udtuULucovjJ",
"colab_type": "text"
},
"source": [
"### Import Titanic dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "7_iYLO3Uv11S",
"colab_type": "code",
"colab": {}
},
"source": [
"df_train = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/train.csv') # training data\n",
"df_test = pd.read_csv('https://storage.googleapis.com/tf-datasets/titanic/eval.csv') # testing data"
],
"execution_count": 88,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "JvRPg8hqoab1",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 197
},
"outputId": "cb8ca95c-86c8-410e-f472-dd40a75f2c73"
},
"source": [
"# see how the dataset looks like\n",
"df_train.head()"
],
"execution_count": 89,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>survived</th>\n",
" <th>sex</th>\n",
" <th>age</th>\n",
" <th>n_siblings_spouses</th>\n",
" <th>parch</th>\n",
" <th>fare</th>\n",
" <th>class</th>\n",
" <th>deck</th>\n",
" <th>embark_town</th>\n",
" <th>alone</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>0</th>\n",
" <td>0</td>\n",
" <td>male</td>\n",
" <td>22.0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>7.2500</td>\n",
" <td>Third</td>\n",
" <td>unknown</td>\n",
" <td>Southampton</td>\n",
" <td>n</td>\n",
" </tr>\n",
" <tr>\n",
" <th>1</th>\n",
" <td>1</td>\n",
" <td>female</td>\n",
" <td>38.0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>71.2833</td>\n",
" <td>First</td>\n",
" <td>C</td>\n",
" <td>Cherbourg</td>\n",
" <td>n</td>\n",
" </tr>\n",
" <tr>\n",
" <th>2</th>\n",
" <td>1</td>\n",
" <td>female</td>\n",
" <td>26.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>7.9250</td>\n",
" <td>Third</td>\n",
" <td>unknown</td>\n",
" <td>Southampton</td>\n",
" <td>y</td>\n",
" </tr>\n",
" <tr>\n",
" <th>3</th>\n",
" <td>1</td>\n",
" <td>female</td>\n",
" <td>35.0</td>\n",
" <td>1</td>\n",
" <td>0</td>\n",
" <td>53.1000</td>\n",
" <td>First</td>\n",
" <td>C</td>\n",
" <td>Southampton</td>\n",
" <td>n</td>\n",
" </tr>\n",
" <tr>\n",
" <th>4</th>\n",
" <td>0</td>\n",
" <td>male</td>\n",
" <td>28.0</td>\n",
" <td>0</td>\n",
" <td>0</td>\n",
" <td>8.4583</td>\n",
" <td>Third</td>\n",
" <td>unknown</td>\n",
" <td>Queenstown</td>\n",
" <td>y</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" survived sex age ... deck embark_town alone\n",
"0 0 male 22.0 ... unknown Southampton n\n",
"1 1 female 38.0 ... C Cherbourg n\n",
"2 1 female 26.0 ... unknown Southampton y\n",
"3 1 female 35.0 ... C Southampton n\n",
"4 0 male 28.0 ... unknown Queenstown y\n",
"\n",
"[5 rows x 10 columns]"
]
},
"metadata": {
"tags": []
},
"execution_count": 89
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "adGXjAQwxPMx",
"colab_type": "code",
"colab": {}
},
"source": [
"y_train = df_train.pop('survived')\n",
"y_test = df_test.pop('survived')"
],
"execution_count": 90,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "BRhb9kyAyXLJ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 287
},
"outputId": "9c732f99-e896-46b5-b8c6-2f71090219e6"
},
"source": [
"df_train.describe()"
],
"execution_count": 91,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>age</th>\n",
" <th>n_siblings_spouses</th>\n",
" <th>parch</th>\n",
" <th>fare</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" <tr>\n",
" <th>count</th>\n",
" <td>627.000000</td>\n",
" <td>627.000000</td>\n",
" <td>627.000000</td>\n",
" <td>627.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>mean</th>\n",
" <td>29.631308</td>\n",
" <td>0.545455</td>\n",
" <td>0.379585</td>\n",
" <td>34.385399</td>\n",
" </tr>\n",
" <tr>\n",
" <th>std</th>\n",
" <td>12.511818</td>\n",
" <td>1.151090</td>\n",
" <td>0.792999</td>\n",
" <td>54.597730</td>\n",
" </tr>\n",
" <tr>\n",
" <th>min</th>\n",
" <td>0.750000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" </tr>\n",
" <tr>\n",
" <th>25%</th>\n",
" <td>23.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>7.895800</td>\n",
" </tr>\n",
" <tr>\n",
" <th>50%</th>\n",
" <td>28.000000</td>\n",
" <td>0.000000</td>\n",
" <td>0.000000</td>\n",
" <td>15.045800</td>\n",
" </tr>\n",
" <tr>\n",
" <th>75%</th>\n",
" <td>35.000000</td>\n",
" <td>1.000000</td>\n",
" <td>0.000000</td>\n",
" <td>31.387500</td>\n",
" </tr>\n",
" <tr>\n",
" <th>max</th>\n",
" <td>80.000000</td>\n",
" <td>8.000000</td>\n",
" <td>5.000000</td>\n",
" <td>512.329200</td>\n",
" </tr>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
" age n_siblings_spouses parch fare\n",
"count 627.000000 627.000000 627.000000 627.000000\n",
"mean 29.631308 0.545455 0.379585 34.385399\n",
"std 12.511818 1.151090 0.792999 54.597730\n",
"min 0.750000 0.000000 0.000000 0.000000\n",
"25% 23.000000 0.000000 0.000000 7.895800\n",
"50% 28.000000 0.000000 0.000000 15.045800\n",
"75% 35.000000 1.000000 0.000000 31.387500\n",
"max 80.000000 8.000000 5.000000 512.329200"
]
},
"metadata": {
"tags": []
},
"execution_count": 91
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "umu-hX_53lkE",
"colab_type": "code",
"colab": {}
},
"source": [
"# Categorical features of the training/testing dataset\n",
"cat_col = ['sex', 'n_siblings_spouses', 'parch', 'class', 'deck', 'embark_town', 'alone']\n",
"\n",
"# Numerical features of the training/testing dataset\n",
"num_col = ['age', 'fare']"
],
"execution_count": 92,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "HG2dTC_m6P8P",
"colab_type": "text"
},
"source": [
"### Create Base Feature Columns\n",
"\n",
"Estimators use a system called feature columns to describe how the model should interpret each of the raw input features. An Estimator expects a vector of numeric inputs, and feature columns describe how the model should convert each feature.\n",
"\n",
"Selecting and crafting the right set of feature columns is key to learning an effective model. A feature column can be either one of the raw inputs in the original features `dict` (a base feature column), or any new columns created using transformations defined over one or multiple base columns (a derived feature columns).\n",
"\n",
"The linear estimator uses both numeric and categorical features. Feature columns work with all TensorFlow estimators and their purpose is to define the features used for modeling. Additionally, they provide some feature engineering capabilities like one-hot-encoding, normalization, and bucketization."
]
},
{
"cell_type": "code",
"metadata": {
"id": "DFTMq7Xp6PXr",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 336
},
"outputId": "f040b8de-acb4-4161-9064-3a570f0b1485"
},
"source": [
"# to read more : https://www.tensorflow.org/api_docs/python/tf/feature_column \n",
"feature_cols = []\n",
"\n",
"for feature in cat_col:\n",
" # create a vocabulary that contains list of unique values in the data column\n",
" vocab = df_train[feature].unique()\n",
" # create a feature column based on the categorical values in the data column\n",
" # append it two a separate feature column\n",
" feature_cols.append(tf.feature_column.categorical_column_with_vocabulary_list(feature, vocab))\n",
"\n",
"# to read more : https://www.tensorflow.org/api_docs/python/tf/feature_column/numeric_column\n",
"for feature in num_col:\n",
" feature_cols.append(tf.feature_column.numeric_column(feature, dtype=tf.float64))\n",
"\n",
"# Show the base feature column\n",
"for f in feature_cols:\n",
" print(f, '\\n')"
],
"execution_count": 93,
"outputs": [
{
"output_type": "stream",
"text": [
"VocabularyListCategoricalColumn(key='sex', vocabulary_list=('male', 'female'), dtype=tf.string, default_value=-1, num_oov_buckets=0) \n",
"\n",
"VocabularyListCategoricalColumn(key='n_siblings_spouses', vocabulary_list=(1, 0, 3, 4, 2, 5, 8), dtype=tf.int64, default_value=-1, num_oov_buckets=0) \n",
"\n",
"VocabularyListCategoricalColumn(key='parch', vocabulary_list=(0, 1, 2, 5, 3, 4), dtype=tf.int64, default_value=-1, num_oov_buckets=0) \n",
"\n",
"VocabularyListCategoricalColumn(key='class', vocabulary_list=('Third', 'First', 'Second'), dtype=tf.string, default_value=-1, num_oov_buckets=0) \n",
"\n",
"VocabularyListCategoricalColumn(key='deck', vocabulary_list=('unknown', 'C', 'G', 'A', 'B', 'D', 'F', 'E'), dtype=tf.string, default_value=-1, num_oov_buckets=0) \n",
"\n",
"VocabularyListCategoricalColumn(key='embark_town', vocabulary_list=('Southampton', 'Cherbourg', 'Queenstown', 'unknown'), dtype=tf.string, default_value=-1, num_oov_buckets=0) \n",
"\n",
"VocabularyListCategoricalColumn(key='alone', vocabulary_list=('n', 'y'), dtype=tf.string, default_value=-1, num_oov_buckets=0) \n",
"\n",
"NumericColumn(key='age', shape=(1,), default_value=None, dtype=tf.float64, normalizer_fn=None) \n",
"\n",
"NumericColumn(key='fare', shape=(1,), default_value=None, dtype=tf.float64, normalizer_fn=None) \n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Zb7TFCSaA846",
"colab_type": "text"
},
"source": [
"### Define a Input Function"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Hanfq8D6I8Jp",
"colab_type": "text"
},
"source": [
"A `input_function` specifies how data is converted to a `tf.data.Dataset` that feeds the input pipeline in a streaming fashion. `tf.data.Dataset` can take in multiple sources such as a dataframe, a csv-formatted file, and more."
]
},
{
"cell_type": "code",
"metadata": {
"id": "FZ99gqeb85mp",
"colab_type": "code",
"colab": {}
},
"source": [
"def make_input_func(data_df, target_df, num_epochs=10, shuffle=True, batch_size=64):\n",
" def input_function():\n",
" # create a tensorflow dataset\n",
" dataset = tf.data.Dataset.from_tensor_slices((dict(data_df), target_df))\n",
"\n",
" # shuffle the dataset\n",
" # to know more : https://www.tensorflow.org/api_docs/python/tf/data/Dataset#shuffle\n",
" if shuffle:\n",
" dataset.shuffle(1000)\n",
" \n",
" # divide the dataset into many mini-batches\n",
" # then populate each mini-batch repeating elements by number of epochs\n",
" # to know more : https://www.tensorflow.org/api_docs/python/tf/data/Dataset#batch\n",
" batch = dataset.batch(batch_size).repeat(num_epochs)\n",
"\n",
" # return the dataset\n",
" return batch\n",
" return input_function\n",
"\n",
"# create a training input function\n",
"train_input_fn = make_input_func(df_train, y_train)\n",
"\n",
"# create a testing input function\n",
"# we don't need to shuffle the testing dataset\n",
"# coz it's for testing purposes\n",
"test_input_fn = make_input_func(df_train, y_train, num_epochs=1, shuffle=False)"
],
"execution_count": 94,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "UEqv94YSTSuD",
"colab_type": "text"
},
"source": [
"### Creating a Linear Estimator Model"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ciNeYg_ZT9gp",
"colab_type": "text"
},
"source": [
"After adding all the base features to the model, let's train the model. Training a model is just a single command using the `tf.estimator` API:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "KW7QKvNBRQOE",
"colab_type": "code",
"colab": {}
},
"source": [
"# Create the linear model based on the base feature columns\n",
"linear_est = tf.estimator.LinearClassifier(feature_columns=feature_cols)\n",
"\n",
"# train the estimator on the dataset\n",
"# here dataset would be the each mini-batch created by the train_input_fn\n",
"linear_est.train(train_input_fn)\n",
"\n",
"# using the trainned estimator evaluate the testing dataset\n",
"# here too, evaluation would be performed on each mini-batch of the testing dataset\n",
"result = linear_est.evaluate(test_input_fn)\n",
"\n",
"# clear the output of the cell\n",
"# since this output is not of any use\n",
"clear_output()"
],
"execution_count": 95,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "x761ULU8aSj4",
"colab_type": "text"
},
"source": [
"Print the Accuracy of the model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "jS_3ido6aXAE",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "919f466c-16df-41bf-fd1d-31f8d505784f"
},
"source": [
"# display the result\n",
"print(\"Accuracy : %.2f \"%(result['accuracy']*100))"
],
"execution_count": 96,
"outputs": [
{
"output_type": "stream",
"text": [
"Accuracy : 80.22 \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VLnM0mgnWhNt",
"colab_type": "text"
},
"source": [
"So, from this model we can get 80% accuracy\n",
"\n",
"We can try to obtain various accuracy values by changing the `batch_size` and `num_epochs`.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "9xDUwafYgMM2",
"colab_type": "text"
},
"source": [
"**Note:** Only run this code block if you have enough time to spend/waste (1-2 hrs) to find out the values.\n",
"\n",
"Also running this code in your local machine can cluter your RAM (<16 GB)"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2dM38jBwwTkx",
"colab_type": "code",
"colab": {}
},
"source": [
"#@title Model Checker {display-mode: \"form\"}\n",
"\n",
"# This code will be hidden when the notebook is loaded.\n",
"\n",
"# don't execute\n",
"\n",
"batches = [32, 64, 96, 128, 144]\n",
"epochs = [10, 100, 1000, 10000]\n",
"Accuracy = dict()\n",
"\n",
"for batch in batches:\n",
" for num_epoch in epochs:\n",
" train_input_fn = make_input_func(df_train, y_train, num_epochs=num_epoch, batch_size=batch)\n",
" test_input_fn = make_input_func(df_train, y_train, num_epochs=1, shuffle=False)\n",
"\n",
" linear_est.train(train_input_fn)\n",
" result = linear_est.evaluate(test_input_fn)\n",
"\n",
" clear_output()\n",
"\n",
" print(\"Batch Size: \", batch)\n",
" print(\"Number of Epochs\", num_epoch)\n",
" print(\"Accuracy : %.2f \"%(result['accuracy']*100))\n",
"\n",
" id = 'B: ' + str(batch) + ' E: ' + str(num_epoch)\n",
" Accuracy.update({id: result['accuracy']*100})\n",
"\n",
"clear_output()\n",
"print(\"Model Checker Completed\")\n",
"\n",
"for key, value in Accuracy.items():\n",
" print(key, value)"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "iym4DBR4h7MV",
"colab_type": "text"
},
"source": [
"### Use the model for prediction"
]
},
{
"cell_type": "code",
"metadata": {
"id": "hTDWZSsodg1v",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 53
},
"outputId": "0a8d5ae5-9007-443a-a6ea-eda5b5d8a465"
},
"source": [
"pred = list(linear_est.predict(test_input_fn))\n",
"clear_output()\n",
"print(pred)"
],
"execution_count": 98,
"outputs": [
{
"output_type": "stream",
"text": [
"[{'logits': array([-2.3956022], dtype=float32), 'logistic': array([0.08350867], dtype=float32), 'probabilities': array([0.9164914 , 0.08350867], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.385266], dtype=float32), 'logistic': array([0.9156968], dtype=float32), 'probabilities': array([0.08430316, 0.9156968 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.3139953], dtype=float32), 'logistic': array([0.5778602], dtype=float32), 'probabilities': array([0.42213982, 0.5778602 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.6583393], dtype=float32), 'logistic': array([0.840015], dtype=float32), 'probabilities': array([0.15998507, 0.840015 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4866104], dtype=float32), 'logistic': array([0.07680219], dtype=float32), 'probabilities': array([0.9231978 , 0.07680218], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5447931], dtype=float32), 'logistic': array([0.07277707], dtype=float32), 'probabilities': array([0.9272229 , 0.07277707], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.06145447], dtype=float32), 'logistic': array([0.51535875], dtype=float32), 'probabilities': array([0.4846412 , 0.51535875], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.658158], dtype=float32), 'logistic': array([0.934512], dtype=float32), 'probabilities': array([0.06548796, 0.934512 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.6333369], dtype=float32), 'logistic': array([0.83662623], dtype=float32), 'probabilities': array([0.16337374, 0.83662623], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4253259], dtype=float32), 'logistic': array([0.08126175], dtype=float32), 'probabilities': array([0.9187383 , 0.08126175], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3621705], dtype=float32), 'logistic': array([0.03349888], dtype=float32), 'probabilities': array([0.9665011 , 0.03349888], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.81703705], dtype=float32), 'logistic': array([0.69360703], dtype=float32), 'probabilities': array([0.30639294, 0.69360703], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8063811], dtype=float32), 'logistic': array([0.14107606], dtype=float32), 'probabilities': array([0.8589239 , 0.14107606], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8856239], dtype=float32), 'logistic': array([0.13174422], dtype=float32), 'probabilities': array([0.8682558 , 0.13174422], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.2705683], dtype=float32), 'logistic': array([0.56723243], dtype=float32), 'probabilities': array([0.43276757, 0.56723243], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.9913704], dtype=float32), 'logistic': array([0.7293585], dtype=float32), 'probabilities': array([0.27064148, 0.7293585 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.1162727], dtype=float32), 'logistic': array([0.10752524], dtype=float32), 'probabilities': array([0.8924748 , 0.10752523], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.7778502], dtype=float32), 'logistic': array([0.3147834], dtype=float32), 'probabilities': array([0.6852166 , 0.31478336], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.328057], dtype=float32), 'logistic': array([0.41871348], dtype=float32), 'probabilities': array([0.58128655, 0.41871348], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0002491], dtype=float32), 'logistic': array([0.11917676], dtype=float32), 'probabilities': array([0.8808232 , 0.11917677], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.0128727], dtype=float32), 'logistic': array([0.26641804], dtype=float32), 'probabilities': array([0.73358196, 0.266418 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.5022029], dtype=float32), 'logistic': array([0.6229769], dtype=float32), 'probabilities': array([0.37702307, 0.6229769 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.99719244], dtype=float32), 'logistic': array([0.2694938], dtype=float32), 'probabilities': array([0.7305062, 0.2694938], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.735545], dtype=float32), 'logistic': array([0.97669584], dtype=float32), 'probabilities': array([0.02330413, 0.97669584], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50157696], dtype=float32), 'logistic': array([0.6228299], dtype=float32), 'probabilities': array([0.37717015, 0.6228299 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.4917898], dtype=float32), 'logistic': array([0.02954674], dtype=float32), 'probabilities': array([0.9704533 , 0.02954674], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.11246891], dtype=float32), 'logistic': array([0.47191238], dtype=float32), 'probabilities': array([0.5280876 , 0.47191238], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6106104], dtype=float32), 'logistic': array([0.16650389], dtype=float32), 'probabilities': array([0.8334961 , 0.16650389], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0002286], dtype=float32), 'logistic': array([0.11917893], dtype=float32), 'probabilities': array([0.8808211 , 0.11917892], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.7156135], dtype=float32), 'logistic': array([0.84756297], dtype=float32), 'probabilities': array([0.15243706, 0.84756297], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.14827865], dtype=float32), 'logistic': array([0.4629981], dtype=float32), 'probabilities': array([0.5370019 , 0.46299812], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.3042066], dtype=float32), 'logistic': array([0.78654206], dtype=float32), 'probabilities': array([0.21345791, 0.78654206], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9969987], dtype=float32), 'logistic': array([0.11951841], dtype=float32), 'probabilities': array([0.8804816 , 0.11951841], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.0007672], dtype=float32), 'logistic': array([0.95260876], dtype=float32), 'probabilities': array([0.04739122, 0.95260876], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3349917], dtype=float32), 'logistic': array([0.08826613], dtype=float32), 'probabilities': array([0.91173387, 0.08826613], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.1067624], dtype=float32), 'logistic': array([0.10844129], dtype=float32), 'probabilities': array([0.8915587 , 0.10844128], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.81493276], dtype=float32), 'logistic': array([0.69315964], dtype=float32), 'probabilities': array([0.30684033, 0.69315964], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2372706], dtype=float32), 'logistic': array([0.09645315], dtype=float32), 'probabilities': array([0.90354687, 0.09645315], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.6111088], dtype=float32), 'logistic': array([0.9315731], dtype=float32), 'probabilities': array([0.06842689, 0.9315731 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2445369], dtype=float32), 'logistic': array([0.77635276], dtype=float32), 'probabilities': array([0.22364727, 0.77635276], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2487648], dtype=float32), 'logistic': array([0.22291403], dtype=float32), 'probabilities': array([0.77708596, 0.22291404], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2424186], dtype=float32), 'logistic': array([0.22401527], dtype=float32), 'probabilities': array([0.77598476, 0.22401528], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.3875228], dtype=float32), 'logistic': array([0.8001965], dtype=float32), 'probabilities': array([0.19980352, 0.8001965 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0212033], dtype=float32), 'logistic': array([0.11699462], dtype=float32), 'probabilities': array([0.88300544, 0.11699463], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6224706], dtype=float32), 'logistic': array([0.06770618], dtype=float32), 'probabilities': array([0.9322938 , 0.06770618], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7485363], dtype=float32), 'logistic': array([0.1482319], dtype=float32), 'probabilities': array([0.85176814, 0.1482319 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.3519194], dtype=float32), 'logistic': array([0.9130867], dtype=float32), 'probabilities': array([0.08691333, 0.9130867 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6055837], dtype=float32), 'logistic': array([0.16720267], dtype=float32), 'probabilities': array([0.83279735, 0.16720268], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9083207], dtype=float32), 'logistic': array([0.05174378], dtype=float32), 'probabilities': array([0.9482562 , 0.05174377], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7057766], dtype=float32), 'logistic': array([0.15371232], dtype=float32), 'probabilities': array([0.84628767, 0.15371232], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3828526], dtype=float32), 'logistic': array([0.08448966], dtype=float32), 'probabilities': array([0.91551036, 0.08448965], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.13174719], dtype=float32), 'logistic': array([0.46711072], dtype=float32), 'probabilities': array([0.53288925, 0.46711072], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8506024], dtype=float32), 'logistic': array([0.05465019], dtype=float32), 'probabilities': array([0.9453498 , 0.05465018], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0655327], dtype=float32), 'logistic': array([0.11249227], dtype=float32), 'probabilities': array([0.88750774, 0.11249228], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2988261], dtype=float32), 'logistic': array([0.21436265], dtype=float32), 'probabilities': array([0.7856373 , 0.21436265], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6939638], dtype=float32), 'logistic': array([0.06333049], dtype=float32), 'probabilities': array([0.9366695 , 0.06333049], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2044327], dtype=float32), 'logistic': array([0.09935313], dtype=float32), 'probabilities': array([0.90064687, 0.09935313], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.89448565], dtype=float32), 'logistic': array([0.290185], dtype=float32), 'probabilities': array([0.709815, 0.290185], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.16824764], dtype=float32), 'logistic': array([0.541963], dtype=float32), 'probabilities': array([0.458037, 0.541963], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5046203], dtype=float32), 'logistic': array([0.07553492], dtype=float32), 'probabilities': array([0.92446506, 0.07553492], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7958386], dtype=float32), 'logistic': array([0.05754947], dtype=float32), 'probabilities': array([0.9424505 , 0.05754946], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50175864], dtype=float32), 'logistic': array([0.62287253], dtype=float32), 'probabilities': array([0.37712744, 0.62287253], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.164569], dtype=float32), 'logistic': array([0.23783804], dtype=float32), 'probabilities': array([0.7621619 , 0.23783804], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.976288], dtype=float32), 'logistic': array([0.12171509], dtype=float32), 'probabilities': array([0.8782849, 0.1217151], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.8109518], dtype=float32), 'logistic': array([0.8594769], dtype=float32), 'probabilities': array([0.14052312, 0.8594769 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.593121], dtype=float32), 'logistic': array([0.06958245], dtype=float32), 'probabilities': array([0.93041754, 0.06958245], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.54794425], dtype=float32), 'logistic': array([0.3663415], dtype=float32), 'probabilities': array([0.63365847, 0.36634147], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-4.0652037], dtype=float32), 'logistic': array([0.01687001], dtype=float32), 'probabilities': array([0.98313004, 0.01687001], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.82108], dtype=float32), 'logistic': array([0.13930434], dtype=float32), 'probabilities': array([0.8606956 , 0.13930434], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6144325], dtype=float32), 'logistic': array([0.64895123], dtype=float32), 'probabilities': array([0.35104874, 0.64895123], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.98932207], dtype=float32), 'logistic': array([0.728954], dtype=float32), 'probabilities': array([0.271046, 0.728954], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9568261], dtype=float32), 'logistic': array([0.12381095], dtype=float32), 'probabilities': array([0.8761891 , 0.12381094], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.22995609], dtype=float32), 'logistic': array([0.557237], dtype=float32), 'probabilities': array([0.44276297, 0.557237 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.00116032], dtype=float32), 'logistic': array([0.5002901], dtype=float32), 'probabilities': array([0.4997099, 0.5002901], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9677322], dtype=float32), 'logistic': array([0.0489051], dtype=float32), 'probabilities': array([0.9510949, 0.0489051], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3156126], dtype=float32), 'logistic': array([0.03503945], dtype=float32), 'probabilities': array([0.9649606 , 0.03503945], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7622488], dtype=float32), 'logistic': array([0.0593986], dtype=float32), 'probabilities': array([0.9406014 , 0.05939861], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6985418], dtype=float32), 'logistic': array([0.6678644], dtype=float32), 'probabilities': array([0.3321356, 0.6678644], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9594928], dtype=float32), 'logistic': array([0.12352194], dtype=float32), 'probabilities': array([0.876478 , 0.12352194], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.7102058], dtype=float32), 'logistic': array([0.846863], dtype=float32), 'probabilities': array([0.15313703, 0.846863 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5092235], dtype=float32), 'logistic': array([0.0752141], dtype=float32), 'probabilities': array([0.9247859, 0.0752141], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.4878559], dtype=float32), 'logistic': array([0.8157562], dtype=float32), 'probabilities': array([0.18424375, 0.8157562 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4678805], dtype=float32), 'logistic': array([0.07814077], dtype=float32), 'probabilities': array([0.9218592 , 0.07814077], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-4.272864], dtype=float32), 'logistic': array([0.0137501], dtype=float32), 'probabilities': array([0.9862499, 0.0137501], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7713103], dtype=float32), 'logistic': array([0.14537944], dtype=float32), 'probabilities': array([0.8546205 , 0.14537945], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.3701876], dtype=float32), 'logistic': array([0.7974104], dtype=float32), 'probabilities': array([0.20258951, 0.7974104 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.1095134], dtype=float32), 'logistic': array([0.24796163], dtype=float32), 'probabilities': array([0.7520384 , 0.24796163], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1032293], dtype=float32), 'logistic': array([0.89121664], dtype=float32), 'probabilities': array([0.10878334, 0.89121664], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5975223], dtype=float32), 'logistic': array([0.06929805], dtype=float32), 'probabilities': array([0.9307019 , 0.06929804], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.4792538], dtype=float32), 'logistic': array([0.02990833], dtype=float32), 'probabilities': array([0.9700917 , 0.02990832], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4301715], dtype=float32), 'logistic': array([0.08090071], dtype=float32), 'probabilities': array([0.9190993 , 0.08090071], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.417571], dtype=float32), 'logistic': array([0.39709815], dtype=float32), 'probabilities': array([0.6029019 , 0.39709815], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.90130264], dtype=float32), 'logistic': array([0.2887829], dtype=float32), 'probabilities': array([0.7112171 , 0.28878286], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.5168319], dtype=float32), 'logistic': array([0.9253134], dtype=float32), 'probabilities': array([0.07468659, 0.9253134 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.4171778], dtype=float32), 'logistic': array([0.19510439], dtype=float32), 'probabilities': array([0.8048956 , 0.19510439], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2518775], dtype=float32), 'logistic': array([0.09518763], dtype=float32), 'probabilities': array([0.9048124 , 0.09518764], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.46801513], dtype=float32), 'logistic': array([0.6149138], dtype=float32), 'probabilities': array([0.38508612, 0.6149138 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.8040976], dtype=float32), 'logistic': array([0.6908503], dtype=float32), 'probabilities': array([0.30914968, 0.6908503 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.4809423], dtype=float32), 'logistic': array([0.6179704], dtype=float32), 'probabilities': array([0.38202965, 0.6179704 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.1173494], dtype=float32), 'logistic': array([0.10742196], dtype=float32), 'probabilities': array([0.892578 , 0.10742195], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.4734049], dtype=float32), 'logistic': array([0.18642563], dtype=float32), 'probabilities': array([0.8135743 , 0.18642563], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.8041455], dtype=float32), 'logistic': array([0.6908605], dtype=float32), 'probabilities': array([0.30913943, 0.6908605 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8527462], dtype=float32), 'logistic': array([0.05453954], dtype=float32), 'probabilities': array([0.94546044, 0.05453953], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.9145062], dtype=float32), 'logistic': array([0.01956017], dtype=float32), 'probabilities': array([0.98043984, 0.01956016], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.4801595], dtype=float32), 'logistic': array([0.02988205], dtype=float32), 'probabilities': array([0.9701179 , 0.02988205], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7644894], dtype=float32), 'logistic': array([0.05927354], dtype=float32), 'probabilities': array([0.9407264 , 0.05927354], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2089716], dtype=float32), 'logistic': array([0.22988307], dtype=float32), 'probabilities': array([0.7701169 , 0.22988304], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.004881], dtype=float32), 'logistic': array([0.73201716], dtype=float32), 'probabilities': array([0.26798284, 0.73201716], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7579482], dtype=float32), 'logistic': array([0.05963933], dtype=float32), 'probabilities': array([0.9403607 , 0.05963933], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3676736], dtype=float32), 'logistic': array([0.03332116], dtype=float32), 'probabilities': array([0.96667886, 0.03332116], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.306526], dtype=float32), 'logistic': array([0.03534799], dtype=float32), 'probabilities': array([0.964652 , 0.03534799], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6783512], dtype=float32), 'logistic': array([0.06426295], dtype=float32), 'probabilities': array([0.9357371 , 0.06426296], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.296512], dtype=float32), 'logistic': array([0.09141225], dtype=float32), 'probabilities': array([0.90858775, 0.09141225], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9855784], dtype=float32), 'logistic': array([0.12072544], dtype=float32), 'probabilities': array([0.87927455, 0.12072543], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.1378963], dtype=float32), 'logistic': array([0.95842916], dtype=float32), 'probabilities': array([0.04157085, 0.95842916], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5261686], dtype=float32), 'logistic': array([0.07404391], dtype=float32), 'probabilities': array([0.925956 , 0.07404391], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8902787], dtype=float32), 'logistic': array([0.1312127], dtype=float32), 'probabilities': array([0.8687873, 0.1312127], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.08677], dtype=float32), 'logistic': array([0.11038937], dtype=float32), 'probabilities': array([0.8896106 , 0.11038935], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.61418], dtype=float32), 'logistic': array([0.02623233], dtype=float32), 'probabilities': array([0.97376764, 0.02623233], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.5581039], dtype=float32), 'logistic': array([0.82608116], dtype=float32), 'probabilities': array([0.17391889, 0.82608116], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9695214], dtype=float32), 'logistic': array([0.12244029], dtype=float32), 'probabilities': array([0.87755966, 0.12244029], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.1355128], dtype=float32), 'logistic': array([0.04166592], dtype=float32), 'probabilities': array([0.9583341 , 0.04166592], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.37605423], dtype=float32), 'logistic': array([0.40707892], dtype=float32), 'probabilities': array([0.5929211, 0.4070789], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.111026], dtype=float32), 'logistic': array([0.24767964], dtype=float32), 'probabilities': array([0.75232035, 0.24767964], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6740875], dtype=float32), 'logistic': array([0.06451982], dtype=float32), 'probabilities': array([0.9354802 , 0.06451982], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.0790519], dtype=float32), 'logistic': array([0.7463145], dtype=float32), 'probabilities': array([0.25368547, 0.7463145 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6566277], dtype=float32), 'logistic': array([0.65850246], dtype=float32), 'probabilities': array([0.34149754, 0.65850246], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9772738], dtype=float32), 'logistic': array([0.12160975], dtype=float32), 'probabilities': array([0.8783903 , 0.12160975], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0972533], dtype=float32), 'logistic': array([0.0432207], dtype=float32), 'probabilities': array([0.95677924, 0.04322069], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.93820065], dtype=float32), 'logistic': array([0.71873605], dtype=float32), 'probabilities': array([0.28126395, 0.71873605], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.7247912], dtype=float32), 'logistic': array([0.67366123], dtype=float32), 'probabilities': array([0.3263388 , 0.67366123], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.08573056], dtype=float32), 'logistic': array([0.47858045], dtype=float32), 'probabilities': array([0.5214195, 0.4785805], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.3712142], dtype=float32), 'logistic': array([0.9146057], dtype=float32), 'probabilities': array([0.08539426, 0.9146057 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.359585], dtype=float32), 'logistic': array([0.9136931], dtype=float32), 'probabilities': array([0.08630691, 0.9136931 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50157696], dtype=float32), 'logistic': array([0.6228299], dtype=float32), 'probabilities': array([0.37717015, 0.6228299 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2737905], dtype=float32), 'logistic': array([0.7813909], dtype=float32), 'probabilities': array([0.21860906, 0.7813909 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7538898], dtype=float32), 'logistic': array([0.05986735], dtype=float32), 'probabilities': array([0.9401326 , 0.05986734], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3676736], dtype=float32), 'logistic': array([0.03332116], dtype=float32), 'probabilities': array([0.96667886, 0.03332116], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0201395], dtype=float32), 'logistic': array([0.04652429], dtype=float32), 'probabilities': array([0.9534757 , 0.04652429], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.5179659], dtype=float32), 'logistic': array([0.82023877], dtype=float32), 'probabilities': array([0.17976125, 0.82023877], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7734177], dtype=float32), 'logistic': array([0.05877765], dtype=float32), 'probabilities': array([0.9412223 , 0.05877765], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8603247], dtype=float32), 'logistic': array([0.13466519], dtype=float32), 'probabilities': array([0.86533475, 0.13466519], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.004962], dtype=float32), 'logistic': array([0.732033], dtype=float32), 'probabilities': array([0.26796696, 0.732033 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.53837556], dtype=float32), 'logistic': array([0.36856556], dtype=float32), 'probabilities': array([0.63143444, 0.36856556], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.8511191], dtype=float32), 'logistic': array([0.70080185], dtype=float32), 'probabilities': array([0.29919815, 0.70080185], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5131], dtype=float32), 'logistic': array([0.07494492], dtype=float32), 'probabilities': array([0.92505515, 0.07494492], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3725448], dtype=float32), 'logistic': array([0.08529039], dtype=float32), 'probabilities': array([0.9147096, 0.0852904], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.5432713], dtype=float32), 'logistic': array([0.97189415], dtype=float32), 'probabilities': array([0.02810579, 0.97189415], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.2720465], dtype=float32), 'logistic': array([0.56759524], dtype=float32), 'probabilities': array([0.43240473, 0.56759524], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2875702], dtype=float32), 'logistic': array([0.09215764], dtype=float32), 'probabilities': array([0.9078424 , 0.09215764], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9816355], dtype=float32), 'logistic': array([0.12114462], dtype=float32), 'probabilities': array([0.8788554 , 0.12114462], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.257531], dtype=float32), 'logistic': array([0.09470184], dtype=float32), 'probabilities': array([0.9052982 , 0.09470184], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.725737], dtype=float32), 'logistic': array([0.02352841], dtype=float32), 'probabilities': array([0.9764716 , 0.02352841], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5029242], dtype=float32), 'logistic': array([0.07565343], dtype=float32), 'probabilities': array([0.92434657, 0.07565343], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.5201992], dtype=float32), 'logistic': array([0.1794322], dtype=float32), 'probabilities': array([0.8205678 , 0.17943218], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4501767], dtype=float32), 'logistic': array([0.07942563], dtype=float32), 'probabilities': array([0.92057437, 0.07942563], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.4661366], dtype=float32), 'logistic': array([0.18753055], dtype=float32), 'probabilities': array([0.8124694 , 0.18753053], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.6225607], dtype=float32), 'logistic': array([0.3491993], dtype=float32), 'probabilities': array([0.6508007, 0.3491993], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.8055233], dtype=float32), 'logistic': array([0.85881996], dtype=float32), 'probabilities': array([0.14118005, 0.85881996], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.804197], dtype=float32), 'logistic': array([0.0570978], dtype=float32), 'probabilities': array([0.9429022, 0.0570978], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.1836119], dtype=float32), 'logistic': array([0.03978712], dtype=float32), 'probabilities': array([0.9602129 , 0.03978711], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.22828072], dtype=float32), 'logistic': array([0.5568236], dtype=float32), 'probabilities': array([0.4431764, 0.5568236], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3763134], dtype=float32), 'logistic': array([0.08499683], dtype=float32), 'probabilities': array([0.9150032 , 0.08499683], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.5201992], dtype=float32), 'logistic': array([0.1794322], dtype=float32), 'probabilities': array([0.8205678 , 0.17943218], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0988805], dtype=float32), 'logistic': array([0.10920568], dtype=float32), 'probabilities': array([0.8907943 , 0.10920567], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.1438975], dtype=float32), 'logistic': array([0.75839454], dtype=float32), 'probabilities': array([0.24160549, 0.75839454], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6566277], dtype=float32), 'logistic': array([0.65850246], dtype=float32), 'probabilities': array([0.34149754, 0.65850246], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5137055], dtype=float32), 'logistic': array([0.07490294], dtype=float32), 'probabilities': array([0.92509705, 0.07490294], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0841463], dtype=float32), 'logistic': array([0.1106473], dtype=float32), 'probabilities': array([0.88935274, 0.1106473 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.55374], dtype=float32), 'logistic': array([0.17454675], dtype=float32), 'probabilities': array([0.8254533 , 0.17454676], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.0549208], dtype=float32), 'logistic': array([0.7417187], dtype=float32), 'probabilities': array([0.2582813, 0.7417187], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7647927], dtype=float32), 'logistic': array([0.05925664], dtype=float32), 'probabilities': array([0.9407434 , 0.05925664], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.55439454], dtype=float32), 'logistic': array([0.63515455], dtype=float32), 'probabilities': array([0.36484545, 0.63515455], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.688309], dtype=float32), 'logistic': array([0.06366675], dtype=float32), 'probabilities': array([0.9363333 , 0.06366675], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.48183388], dtype=float32), 'logistic': array([0.38181916], dtype=float32), 'probabilities': array([0.6181808, 0.3818192], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.7472577], dtype=float32), 'logistic': array([0.9397583], dtype=float32), 'probabilities': array([0.06024171, 0.9397583 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([4.5523725], dtype=float32), 'logistic': array([0.9895678], dtype=float32), 'probabilities': array([0.01043219, 0.9895678 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.33267885], dtype=float32), 'logistic': array([0.582411], dtype=float32), 'probabilities': array([0.41758895, 0.582411 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.422395], dtype=float32), 'logistic': array([0.08148083], dtype=float32), 'probabilities': array([0.9185192 , 0.08148083], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.3515981], dtype=float32), 'logistic': array([0.20560923], dtype=float32), 'probabilities': array([0.7943908 , 0.20560922], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50157696], dtype=float32), 'logistic': array([0.6228299], dtype=float32), 'probabilities': array([0.37717015, 0.6228299 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2333276], dtype=float32), 'logistic': array([0.09679732], dtype=float32), 'probabilities': array([0.9032027 , 0.09679732], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6148093], dtype=float32), 'logistic': array([0.06819138], dtype=float32), 'probabilities': array([0.9318086 , 0.06819138], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5189044], dtype=float32), 'logistic': array([0.07454348], dtype=float32), 'probabilities': array([0.9254565 , 0.07454349], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.1488968], dtype=float32), 'logistic': array([0.75930935], dtype=float32), 'probabilities': array([0.24069065, 0.75930935], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.9407622], dtype=float32), 'logistic': array([0.87443584], dtype=float32), 'probabilities': array([0.12556414, 0.87443584], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2425827], dtype=float32), 'logistic': array([0.22398674], dtype=float32), 'probabilities': array([0.77601326, 0.22398676], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6740763], dtype=float32), 'logistic': array([0.0645205], dtype=float32), 'probabilities': array([0.93547946, 0.06452049], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.67872125], dtype=float32), 'logistic': array([0.6634532], dtype=float32), 'probabilities': array([0.33654675, 0.6634532 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.79683405], dtype=float32), 'logistic': array([0.31070316], dtype=float32), 'probabilities': array([0.68929684, 0.31070316], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50157696], dtype=float32), 'logistic': array([0.6228299], dtype=float32), 'probabilities': array([0.37717015, 0.6228299 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2651176], dtype=float32), 'logistic': array([0.77990586], dtype=float32), 'probabilities': array([0.22009417, 0.77990586], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.48387903], dtype=float32), 'logistic': array([0.38133657], dtype=float32), 'probabilities': array([0.61866343, 0.38133654], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9486164], dtype=float32), 'logistic': array([0.12470432], dtype=float32), 'probabilities': array([0.8752957, 0.1247043], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0161247], dtype=float32), 'logistic': array([0.1175203], dtype=float32), 'probabilities': array([0.88247967, 0.1175203 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.25178558], dtype=float32), 'logistic': array([0.56261593], dtype=float32), 'probabilities': array([0.43738404, 0.56261593], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.761865], dtype=float32), 'logistic': array([0.05942005], dtype=float32), 'probabilities': array([0.9405799 , 0.05942005], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2505047], dtype=float32), 'logistic': array([0.09530593], dtype=float32), 'probabilities': array([0.904694 , 0.09530593], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.8238831], dtype=float32), 'logistic': array([0.30494002], dtype=float32), 'probabilities': array([0.69505996, 0.30494 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.0647948], dtype=float32), 'logistic': array([0.88743407], dtype=float32), 'probabilities': array([0.11256596, 0.88743407], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.8179055], dtype=float32), 'logistic': array([0.30620846], dtype=float32), 'probabilities': array([0.69379157, 0.30620843], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5938683], dtype=float32), 'logistic': array([0.06953409], dtype=float32), 'probabilities': array([0.93046594, 0.06953409], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2666465], dtype=float32), 'logistic': array([0.21983187], dtype=float32), 'probabilities': array([0.7801681 , 0.21983185], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.2711403], dtype=float32), 'logistic': array([0.96342534], dtype=float32), 'probabilities': array([0.03657462, 0.96342534], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50157696], dtype=float32), 'logistic': array([0.6228299], dtype=float32), 'probabilities': array([0.37717015, 0.6228299 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.422384], dtype=float32), 'logistic': array([0.08148165], dtype=float32), 'probabilities': array([0.91851836, 0.08148165], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.561176], dtype=float32), 'logistic': array([0.92832077], dtype=float32), 'probabilities': array([0.07167924, 0.92832077], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.9007814], dtype=float32), 'logistic': array([0.94788504], dtype=float32), 'probabilities': array([0.05211494, 0.94788504], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.448464], dtype=float32), 'logistic': array([0.9691853], dtype=float32), 'probabilities': array([0.0308147, 0.9691853], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.034058], dtype=float32), 'logistic': array([0.26229814], dtype=float32), 'probabilities': array([0.7377019 , 0.26229814], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.7414486], dtype=float32), 'logistic': array([0.97682995], dtype=float32), 'probabilities': array([0.02317013, 0.97682995], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.9125917], dtype=float32), 'logistic': array([0.94846535], dtype=float32), 'probabilities': array([0.0515346 , 0.94846535], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([4.2477703], dtype=float32), 'logistic': array([0.98590547], dtype=float32), 'probabilities': array([0.01409458, 0.98590547], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.5084926], dtype=float32), 'logistic': array([0.8188377], dtype=float32), 'probabilities': array([0.1811623, 0.8188377], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.1950438], dtype=float32), 'logistic': array([0.10019644], dtype=float32), 'probabilities': array([0.8998036 , 0.10019644], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.4542805], dtype=float32), 'logistic': array([0.81065637], dtype=float32), 'probabilities': array([0.18934366, 0.81065637], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.0240715], dtype=float32), 'logistic': array([0.8833014], dtype=float32), 'probabilities': array([0.11669865, 0.8833014 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.9534428], dtype=float32), 'logistic': array([0.98117274], dtype=float32), 'probabilities': array([0.01882726, 0.98117274], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5131], dtype=float32), 'logistic': array([0.07494492], dtype=float32), 'probabilities': array([0.92505515, 0.07494492], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7197144], dtype=float32), 'logistic': array([0.06182003], dtype=float32), 'probabilities': array([0.93818 , 0.06182003], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.6908244], dtype=float32), 'logistic': array([0.8443326], dtype=float32), 'probabilities': array([0.15566745, 0.8443326 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3676736], dtype=float32), 'logistic': array([0.03332116], dtype=float32), 'probabilities': array([0.96667886, 0.03332116], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.6634781], dtype=float32), 'logistic': array([0.93483686], dtype=float32), 'probabilities': array([0.06516314, 0.93483686], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-4.1540074], dtype=float32), 'logistic': array([0.01545865], dtype=float32), 'probabilities': array([0.9845413 , 0.01545865], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.42091304], dtype=float32), 'logistic': array([0.6037017], dtype=float32), 'probabilities': array([0.3962983, 0.6037017], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.40013117], dtype=float32), 'logistic': array([0.5987192], dtype=float32), 'probabilities': array([0.40128085, 0.5987192 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.0037479], dtype=float32), 'logistic': array([0.26820517], dtype=float32), 'probabilities': array([0.73179483, 0.26820517], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.363932], dtype=float32), 'logistic': array([0.91403526], dtype=float32), 'probabilities': array([0.08596474, 0.91403526], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.0161726], dtype=float32), 'logistic': array([0.2657736], dtype=float32), 'probabilities': array([0.7342264, 0.2657736], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.4740448], dtype=float32), 'logistic': array([0.03005983], dtype=float32), 'probabilities': array([0.9699402 , 0.03005983], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.933906], dtype=float32), 'logistic': array([0.12631889], dtype=float32), 'probabilities': array([0.87368107, 0.12631887], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8856239], dtype=float32), 'logistic': array([0.13174422], dtype=float32), 'probabilities': array([0.8682558 , 0.13174422], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7597777], dtype=float32), 'logistic': array([0.14681819], dtype=float32), 'probabilities': array([0.8531818 , 0.14681819], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2212136], dtype=float32), 'logistic': array([0.09786161], dtype=float32), 'probabilities': array([0.9021384 , 0.09786162], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.7063655], dtype=float32), 'logistic': array([0.84636426], dtype=float32), 'probabilities': array([0.15363573, 0.84636426], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.60261077], dtype=float32), 'logistic': array([0.6462534], dtype=float32), 'probabilities': array([0.3537466, 0.6462534], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.38720793], dtype=float32), 'logistic': array([0.59561044], dtype=float32), 'probabilities': array([0.4043896 , 0.59561044], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.1992877], dtype=float32), 'logistic': array([0.23160195], dtype=float32), 'probabilities': array([0.768398 , 0.23160197], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4703276], dtype=float32), 'logistic': array([0.07796468], dtype=float32), 'probabilities': array([0.9220353 , 0.07796468], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0002491], dtype=float32), 'logistic': array([0.11917676], dtype=float32), 'probabilities': array([0.8808232 , 0.11917677], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7538898], dtype=float32), 'logistic': array([0.05986735], dtype=float32), 'probabilities': array([0.9401326 , 0.05986734], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.389589], dtype=float32), 'logistic': array([0.9673776], dtype=float32), 'probabilities': array([0.03262242, 0.9673776 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.68650836], dtype=float32), 'logistic': array([0.66518974], dtype=float32), 'probabilities': array([0.33481026, 0.66518974], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.5022029], dtype=float32), 'logistic': array([0.6229769], dtype=float32), 'probabilities': array([0.37702307, 0.6229769 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.5022029], dtype=float32), 'logistic': array([0.6229769], dtype=float32), 'probabilities': array([0.37702307, 0.6229769 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.410484], dtype=float32), 'logistic': array([0.03196941], dtype=float32), 'probabilities': array([0.9680305 , 0.03196941], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.97407967], dtype=float32), 'logistic': array([0.2740681], dtype=float32), 'probabilities': array([0.72593194, 0.2740681 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.39983422], dtype=float32), 'logistic': array([0.59864783], dtype=float32), 'probabilities': array([0.40135217, 0.59864783], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0594027], dtype=float32), 'logistic': array([0.04481326], dtype=float32), 'probabilities': array([0.9551868 , 0.04481326], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3349917], dtype=float32), 'logistic': array([0.08826613], dtype=float32), 'probabilities': array([0.91173387, 0.08826613], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1425052], dtype=float32), 'logistic': array([0.8949663], dtype=float32), 'probabilities': array([0.10503367, 0.8949663 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.99139065], dtype=float32), 'logistic': array([0.7293625], dtype=float32), 'probabilities': array([0.27063748, 0.7293625 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.50157696], dtype=float32), 'logistic': array([0.6228299], dtype=float32), 'probabilities': array([0.37717015, 0.6228299 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.4116638], dtype=float32), 'logistic': array([0.96806705], dtype=float32), 'probabilities': array([0.03193292, 0.96806705], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2314618], dtype=float32), 'logistic': array([0.09696057], dtype=float32), 'probabilities': array([0.90303946, 0.09696058], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3833773], dtype=float32), 'logistic': array([0.08444908], dtype=float32), 'probabilities': array([0.9155509 , 0.08444907], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.8791506], dtype=float32), 'logistic': array([0.9468061], dtype=float32), 'probabilities': array([0.0531939, 0.9468061], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.19087619], dtype=float32), 'logistic': array([0.5475747], dtype=float32), 'probabilities': array([0.45242533, 0.5475747 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3847098], dtype=float32), 'logistic': array([0.08434611], dtype=float32), 'probabilities': array([0.9156538, 0.0843461], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.878694], dtype=float32), 'logistic': array([0.94678307], dtype=float32), 'probabilities': array([0.0532169 , 0.94678307], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9293168], dtype=float32), 'logistic': array([0.05072322], dtype=float32), 'probabilities': array([0.9492768 , 0.05072321], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.1729797], dtype=float32), 'logistic': array([0.23631682], dtype=float32), 'probabilities': array([0.7636832 , 0.23631683], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2029834], dtype=float32), 'logistic': array([0.09948291], dtype=float32), 'probabilities': array([0.90051717, 0.0994829 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4901433], dtype=float32), 'logistic': array([0.07655207], dtype=float32), 'probabilities': array([0.9234479 , 0.07655206], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.3272507], dtype=float32), 'logistic': array([0.9111089], dtype=float32), 'probabilities': array([0.08889107, 0.9111089 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.65942895], dtype=float32), 'logistic': array([0.3408679], dtype=float32), 'probabilities': array([0.65913206, 0.34086788], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4685066], dtype=float32), 'logistic': array([0.07809568], dtype=float32), 'probabilities': array([0.92190427, 0.07809568], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.8788614], dtype=float32), 'logistic': array([0.97974443], dtype=float32), 'probabilities': array([0.02025558, 0.97974443], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.31261688], dtype=float32), 'logistic': array([0.5775239], dtype=float32), 'probabilities': array([0.4224761, 0.5775239], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5104554], dtype=float32), 'logistic': array([0.07512846], dtype=float32), 'probabilities': array([0.92487156, 0.07512846], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.10390847], dtype=float32), 'logistic': array([0.52595377], dtype=float32), 'probabilities': array([0.4740462 , 0.52595377], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5777085], dtype=float32), 'logistic': array([0.07058692], dtype=float32), 'probabilities': array([0.929413 , 0.07058692], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6879941], dtype=float32), 'logistic': array([0.15603982], dtype=float32), 'probabilities': array([0.84396017, 0.1560398 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.2229578], dtype=float32), 'logistic': array([0.03831086], dtype=float32), 'probabilities': array([0.9616892 , 0.03831086], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6770184], dtype=float32), 'logistic': array([0.06434315], dtype=float32), 'probabilities': array([0.93565685, 0.06434314], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.605623], dtype=float32), 'logistic': array([0.0687774], dtype=float32), 'probabilities': array([0.9312226 , 0.06877741], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.981054], dtype=float32), 'logistic': array([0.12120653], dtype=float32), 'probabilities': array([0.8787934 , 0.12120652], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.553436], dtype=float32), 'logistic': array([0.3650676], dtype=float32), 'probabilities': array([0.6349324, 0.3650676], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4686072], dtype=float32), 'logistic': array([0.07808845], dtype=float32), 'probabilities': array([0.9219116 , 0.07808845], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.6225607], dtype=float32), 'logistic': array([0.3491993], dtype=float32), 'probabilities': array([0.6508007, 0.3491993], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4943633], dtype=float32), 'logistic': array([0.07625429], dtype=float32), 'probabilities': array([0.92374575, 0.07625428], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9486164], dtype=float32), 'logistic': array([0.12470432], dtype=float32), 'probabilities': array([0.8752957, 0.1247043], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.4327016], dtype=float32), 'logistic': array([0.03128894], dtype=float32), 'probabilities': array([0.9687111 , 0.03128894], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.2307033], dtype=float32), 'logistic': array([0.5574214], dtype=float32), 'probabilities': array([0.44257864, 0.5574214 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2043992], dtype=float32), 'logistic': array([0.7693064], dtype=float32), 'probabilities': array([0.23069353, 0.7693064 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2993449], dtype=float32), 'logistic': array([0.78572476], dtype=float32), 'probabilities': array([0.2142753 , 0.78572476], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9695214], dtype=float32), 'logistic': array([0.12244029], dtype=float32), 'probabilities': array([0.87755966, 0.12244029], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9969987], dtype=float32), 'logistic': array([0.11951841], dtype=float32), 'probabilities': array([0.8804816 , 0.11951841], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.196482], dtype=float32), 'logistic': array([0.10006686], dtype=float32), 'probabilities': array([0.89993316, 0.10006686], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.026887], dtype=float32), 'logistic': array([0.11640875], dtype=float32), 'probabilities': array([0.8835913 , 0.11640874], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.5465266], dtype=float32), 'logistic': array([0.8244115], dtype=float32), 'probabilities': array([0.1755885, 0.8244115], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7427081], dtype=float32), 'logistic': array([0.14896928], dtype=float32), 'probabilities': array([0.8510307 , 0.14896928], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2857865], dtype=float32), 'logistic': array([0.21656686], dtype=float32), 'probabilities': array([0.78343314, 0.21656683], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.38720793], dtype=float32), 'logistic': array([0.59561044], dtype=float32), 'probabilities': array([0.4043896 , 0.59561044], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.69920343], dtype=float32), 'logistic': array([0.6680111], dtype=float32), 'probabilities': array([0.33198887, 0.6680111 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.303962], dtype=float32), 'logistic': array([0.09079536], dtype=float32), 'probabilities': array([0.9092046 , 0.09079536], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.74129957], dtype=float32), 'logistic': array([0.32272002], dtype=float32), 'probabilities': array([0.67727995, 0.32272 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.2550628], dtype=float32), 'logistic': array([0.9628546], dtype=float32), 'probabilities': array([0.03714539, 0.9628546 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.1612864], dtype=float32), 'logistic': array([0.7615664], dtype=float32), 'probabilities': array([0.23843361, 0.7615664 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8925749], dtype=float32), 'logistic': array([0.13095115], dtype=float32), 'probabilities': array([0.86904883, 0.13095115], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.023584], dtype=float32), 'logistic': array([0.11674892], dtype=float32), 'probabilities': array([0.8832511 , 0.11674892], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.712678], dtype=float32), 'logistic': array([0.6709927], dtype=float32), 'probabilities': array([0.32900736, 0.6709927 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4183], dtype=float32), 'logistic': array([0.08178784], dtype=float32), 'probabilities': array([0.9182121 , 0.08178784], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.1059955], dtype=float32), 'logistic': array([0.75138175], dtype=float32), 'probabilities': array([0.2486182 , 0.75138175], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7606132], dtype=float32), 'logistic': array([0.05949005], dtype=float32), 'probabilities': array([0.94051 , 0.05949005], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.5158379], dtype=float32), 'logistic': array([0.18007523], dtype=float32), 'probabilities': array([0.8199248 , 0.18007523], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.976057], dtype=float32), 'logistic': array([0.87826025], dtype=float32), 'probabilities': array([0.12173978, 0.87826025], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2734163], dtype=float32), 'logistic': array([0.09334868], dtype=float32), 'probabilities': array([0.9066513 , 0.09334867], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2068355], dtype=float32), 'logistic': array([0.09913834], dtype=float32), 'probabilities': array([0.9008617 , 0.09913834], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.5992047], dtype=float32), 'logistic': array([0.35452566], dtype=float32), 'probabilities': array([0.6454744 , 0.35452566], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.98143727], dtype=float32), 'logistic': array([0.2726067], dtype=float32), 'probabilities': array([0.7273933 , 0.27260667], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0389473], dtype=float32), 'logistic': array([0.11517397], dtype=float32), 'probabilities': array([0.884826 , 0.11517395], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6302459], dtype=float32), 'logistic': array([0.16379666], dtype=float32), 'probabilities': array([0.83620334, 0.16379666], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.6068747], dtype=float32), 'logistic': array([0.9313027], dtype=float32), 'probabilities': array([0.06869729, 0.9313027 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.9171173], dtype=float32), 'logistic': array([0.28554565], dtype=float32), 'probabilities': array([0.71445435, 0.28554565], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0126083], dtype=float32), 'logistic': array([0.04685951], dtype=float32), 'probabilities': array([0.95314056, 0.04685951], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.81726396], dtype=float32), 'logistic': array([0.30634478], dtype=float32), 'probabilities': array([0.69365525, 0.30634475], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.1852489], dtype=float32), 'logistic': array([0.03972463], dtype=float32), 'probabilities': array([0.96027535, 0.03972462], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.43871], dtype=float32), 'logistic': array([0.08026809], dtype=float32), 'probabilities': array([0.9197319 , 0.08026809], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4901636], dtype=float32), 'logistic': array([0.07655063], dtype=float32), 'probabilities': array([0.9234494 , 0.07655063], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1543393], dtype=float32), 'logistic': array([0.8960736], dtype=float32), 'probabilities': array([0.10392642, 0.8960736 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7647927], dtype=float32), 'logistic': array([0.05925664], dtype=float32), 'probabilities': array([0.9407434 , 0.05925664], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.91063017], dtype=float32), 'logistic': array([0.7131291], dtype=float32), 'probabilities': array([0.2868709, 0.7131291], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.7234895], dtype=float32), 'logistic': array([0.93839854], dtype=float32), 'probabilities': array([0.06160144, 0.93839854], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.6978984], dtype=float32), 'logistic': array([0.33227834], dtype=float32), 'probabilities': array([0.6677217 , 0.33227834], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.981054], dtype=float32), 'logistic': array([0.12120653], dtype=float32), 'probabilities': array([0.8787934 , 0.12120652], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.690233], dtype=float32), 'logistic': array([0.06355215], dtype=float32), 'probabilities': array([0.93644786, 0.06355215], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.4284667], dtype=float32), 'logistic': array([0.8066623], dtype=float32), 'probabilities': array([0.19333771, 0.8066623 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5385733], dtype=float32), 'logistic': array([0.0731979], dtype=float32), 'probabilities': array([0.9268021, 0.0731979], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9486164], dtype=float32), 'logistic': array([0.12470432], dtype=float32), 'probabilities': array([0.8752957, 0.1247043], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2300525], dtype=float32), 'logistic': array([0.22617224], dtype=float32), 'probabilities': array([0.7738278 , 0.22617222], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6011258], dtype=float32), 'logistic': array([0.64591384], dtype=float32), 'probabilities': array([0.35408616, 0.64591384], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.6225607], dtype=float32), 'logistic': array([0.3491993], dtype=float32), 'probabilities': array([0.6508007, 0.3491993], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.8371407], dtype=float32), 'logistic': array([0.86261016], dtype=float32), 'probabilities': array([0.13738982, 0.86261016], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.1980975], dtype=float32), 'logistic': array([0.23181383], dtype=float32), 'probabilities': array([0.76818615, 0.23181382], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.670245], dtype=float32), 'logistic': array([0.15839152], dtype=float32), 'probabilities': array([0.84160846, 0.15839152], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.585675], dtype=float32), 'logistic': array([0.07006606], dtype=float32), 'probabilities': array([0.92993397, 0.07006606], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.1920502], dtype=float32), 'logistic': array([0.10046665], dtype=float32), 'probabilities': array([0.89953333, 0.10046665], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4672747], dtype=float32), 'logistic': array([0.07818443], dtype=float32), 'probabilities': array([0.9218155 , 0.07818442], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.3802812], dtype=float32), 'logistic': array([0.9153113], dtype=float32), 'probabilities': array([0.08468877, 0.9153113 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.296512], dtype=float32), 'logistic': array([0.09141225], dtype=float32), 'probabilities': array([0.90858775, 0.09141225], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.7952183], dtype=float32), 'logistic': array([0.6889507], dtype=float32), 'probabilities': array([0.31104928, 0.6889507 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.1393854], dtype=float32), 'logistic': array([0.46520993], dtype=float32), 'probabilities': array([0.53479004, 0.46520996], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.4148956], dtype=float32), 'logistic': array([0.6022612], dtype=float32), 'probabilities': array([0.39773887, 0.6022612 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.7331061], dtype=float32), 'logistic': array([0.6754865], dtype=float32), 'probabilities': array([0.3245135, 0.6754865], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2641455], dtype=float32), 'logistic': array([0.2202611], dtype=float32), 'probabilities': array([0.7797389 , 0.22026108], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.442271], dtype=float32), 'logistic': array([0.08000559], dtype=float32), 'probabilities': array([0.91999435, 0.08000559], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.678146], dtype=float32), 'logistic': array([0.8426588], dtype=float32), 'probabilities': array([0.15734112, 0.8426588 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5958064], dtype=float32), 'logistic': array([0.0694088], dtype=float32), 'probabilities': array([0.9305912, 0.0694088], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.3857218], dtype=float32), 'logistic': array([0.20009163], dtype=float32), 'probabilities': array([0.7999084 , 0.20009165], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2747643], dtype=float32), 'logistic': array([0.78155726], dtype=float32), 'probabilities': array([0.21844277, 0.78155726], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.9508957], dtype=float32), 'logistic': array([0.7212953], dtype=float32), 'probabilities': array([0.2787047, 0.7212953], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9294584], dtype=float32), 'logistic': array([0.0507164], dtype=float32), 'probabilities': array([0.94928366, 0.05071639], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.5125692], dtype=float32), 'logistic': array([0.9250183], dtype=float32), 'probabilities': array([0.07498172, 0.9250183 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5099707], dtype=float32), 'logistic': array([0.07516214], dtype=float32), 'probabilities': array([0.9248379 , 0.07516215], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.604396], dtype=float32), 'logistic': array([0.93114394], dtype=float32), 'probabilities': array([0.06885603, 0.93114394], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0002286], dtype=float32), 'logistic': array([0.11917893], dtype=float32), 'probabilities': array([0.8808211 , 0.11917892], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0144017], dtype=float32), 'logistic': array([0.04677948], dtype=float32), 'probabilities': array([0.9532205 , 0.04677948], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.17100972], dtype=float32), 'logistic': array([0.54264855], dtype=float32), 'probabilities': array([0.45735145, 0.54264855], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.33979052], dtype=float32), 'logistic': array([0.41586033], dtype=float32), 'probabilities': array([0.58413965, 0.41586035], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7215909], dtype=float32), 'logistic': array([0.15166637], dtype=float32), 'probabilities': array([0.84833366, 0.15166637], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.5152626], dtype=float32), 'logistic': array([0.9252049], dtype=float32), 'probabilities': array([0.07479511, 0.9252049 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2831852], dtype=float32), 'logistic': array([0.21700852], dtype=float32), 'probabilities': array([0.78299147, 0.21700852], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.8385616], dtype=float32), 'logistic': array([0.6981622], dtype=float32), 'probabilities': array([0.30183783, 0.6981622 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.14977401], dtype=float32), 'logistic': array([0.53737366], dtype=float32), 'probabilities': array([0.4626263 , 0.53737366], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.8249847], dtype=float32), 'logistic': array([0.8611632], dtype=float32), 'probabilities': array([0.13883683, 0.8611632 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.432692], dtype=float32), 'logistic': array([0.19267957], dtype=float32), 'probabilities': array([0.8073204 , 0.19267957], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.7952807], dtype=float32), 'logistic': array([0.94242024], dtype=float32), 'probabilities': array([0.05757973, 0.94242024], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.1734807], dtype=float32), 'logistic': array([0.9598241], dtype=float32), 'probabilities': array([0.04017598, 0.9598241 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.9257134], dtype=float32), 'logistic': array([0.8727742], dtype=float32), 'probabilities': array([0.12722579, 0.8727742 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.31698674], dtype=float32), 'logistic': array([0.57858974], dtype=float32), 'probabilities': array([0.4214103 , 0.57858974], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.23308939], dtype=float32), 'logistic': array([0.5580099], dtype=float32), 'probabilities': array([0.44199002, 0.5580099 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.93945676], dtype=float32), 'logistic': array([0.2810101], dtype=float32), 'probabilities': array([0.7189899, 0.2810101], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.6640242], dtype=float32), 'logistic': array([0.84077746], dtype=float32), 'probabilities': array([0.15922251, 0.84077746], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.1167802], dtype=float32), 'logistic': array([0.24660902], dtype=float32), 'probabilities': array([0.753391 , 0.24660902], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.654604], dtype=float32), 'logistic': array([0.06570581], dtype=float32), 'probabilities': array([0.93429416, 0.06570581], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.12281876], dtype=float32), 'logistic': array([0.5306662], dtype=float32), 'probabilities': array([0.46933383, 0.5306662 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7806828], dtype=float32), 'logistic': array([0.14421883], dtype=float32), 'probabilities': array([0.85578114, 0.14421885], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4896586], dtype=float32), 'logistic': array([0.07658634], dtype=float32), 'probabilities': array([0.92341363, 0.07658633], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.48106343], dtype=float32), 'logistic': array([0.61799896], dtype=float32), 'probabilities': array([0.38200104, 0.61799896], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.2768235], dtype=float32), 'logistic': array([0.90693927], dtype=float32), 'probabilities': array([0.0930607 , 0.90693927], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.47435695], dtype=float32), 'logistic': array([0.6164144], dtype=float32), 'probabilities': array([0.3835855, 0.6164144], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.964948], dtype=float32), 'logistic': array([0.9509652], dtype=float32), 'probabilities': array([0.04903476, 0.9509652 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.05791729], dtype=float32), 'logistic': array([0.5144753], dtype=float32), 'probabilities': array([0.48552474, 0.5144753 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.265048], dtype=float32), 'logistic': array([0.03678991], dtype=float32), 'probabilities': array([0.96321005, 0.03678991], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6916592], dtype=float32), 'logistic': array([0.06346732], dtype=float32), 'probabilities': array([0.9365326 , 0.06346732], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3841245], dtype=float32), 'logistic': array([0.08439132], dtype=float32), 'probabilities': array([0.9156087 , 0.08439132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.15954393], dtype=float32), 'logistic': array([0.4601984], dtype=float32), 'probabilities': array([0.53980166, 0.4601984 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9296596], dtype=float32), 'logistic': array([0.05070671], dtype=float32), 'probabilities': array([0.9492933 , 0.05070671], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3239946], dtype=float32), 'logistic': array([0.03475714], dtype=float32), 'probabilities': array([0.96524286, 0.03475714], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6013586], dtype=float32), 'logistic': array([0.64596707], dtype=float32), 'probabilities': array([0.3540329 , 0.64596707], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.41451973], dtype=float32), 'logistic': array([0.39782885], dtype=float32), 'probabilities': array([0.6021712 , 0.39782888], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.257531], dtype=float32), 'logistic': array([0.09470184], dtype=float32), 'probabilities': array([0.9052982 , 0.09470184], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.8543033], dtype=float32), 'logistic': array([0.7014691], dtype=float32), 'probabilities': array([0.2985309, 0.7014691], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.5698239], dtype=float32), 'logistic': array([0.8277585], dtype=float32), 'probabilities': array([0.1722415, 0.8277585], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.32636482], dtype=float32), 'logistic': array([0.41912535], dtype=float32), 'probabilities': array([0.5808746 , 0.41912538], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6729586], dtype=float32), 'logistic': array([0.06458799], dtype=float32), 'probabilities': array([0.935412 , 0.06458799], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.7865321], dtype=float32), 'logistic': array([0.31291378], dtype=float32), 'probabilities': array([0.6870862 , 0.31291378], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0590394], dtype=float32), 'logistic': array([0.04482882], dtype=float32), 'probabilities': array([0.9551711 , 0.04482882], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.4927127], dtype=float32), 'logistic': array([0.9236294], dtype=float32), 'probabilities': array([0.07637062, 0.9236294 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.5618186], dtype=float32), 'logistic': array([0.02760357], dtype=float32), 'probabilities': array([0.9723964 , 0.02760356], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.27543885], dtype=float32), 'logistic': array([0.5684276], dtype=float32), 'probabilities': array([0.43157235, 0.5684276 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0826724], dtype=float32), 'logistic': array([0.11079242], dtype=float32), 'probabilities': array([0.88920754, 0.11079242], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.762885], dtype=float32), 'logistic': array([0.05936306], dtype=float32), 'probabilities': array([0.940637 , 0.05936307], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2029068], dtype=float32), 'logistic': array([0.76904154], dtype=float32), 'probabilities': array([0.2309585 , 0.76904154], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.6808465], dtype=float32), 'logistic': array([0.02458213], dtype=float32), 'probabilities': array([0.97541785, 0.02458212], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0002491], dtype=float32), 'logistic': array([0.11917676], dtype=float32), 'probabilities': array([0.8808232 , 0.11917677], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.67277503], dtype=float32), 'logistic': array([0.33787575], dtype=float32), 'probabilities': array([0.6621243 , 0.33787572], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.79312205], dtype=float32), 'logistic': array([0.3114987], dtype=float32), 'probabilities': array([0.68850136, 0.3114987 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9426663], dtype=float32), 'logistic': array([0.05008427], dtype=float32), 'probabilities': array([0.9499157 , 0.05008427], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8455608], dtype=float32), 'logistic': array([0.05491124], dtype=float32), 'probabilities': array([0.9450888 , 0.05491124], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2030567], dtype=float32), 'logistic': array([0.23093188], dtype=float32), 'probabilities': array([0.7690681, 0.2309319], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.2037413], dtype=float32), 'logistic': array([0.900585], dtype=float32), 'probabilities': array([0.09941501, 0.900585 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.8174108], dtype=float32), 'logistic': array([0.8602552], dtype=float32), 'probabilities': array([0.13974483, 0.8602552 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.37055105], dtype=float32), 'logistic': array([0.40840787], dtype=float32), 'probabilities': array([0.5915921 , 0.40840787], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7657616], dtype=float32), 'logistic': array([0.05920263], dtype=float32), 'probabilities': array([0.9407973 , 0.05920264], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.6566277], dtype=float32), 'logistic': array([0.65850246], dtype=float32), 'probabilities': array([0.34149754, 0.65850246], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.054557], dtype=float32), 'logistic': array([0.04502114], dtype=float32), 'probabilities': array([0.9549789 , 0.04502114], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.4686661], dtype=float32), 'logistic': array([0.8128546], dtype=float32), 'probabilities': array([0.18714546, 0.8128546 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7262318], dtype=float32), 'logistic': array([0.06144311], dtype=float32), 'probabilities': array([0.93855697, 0.06144311], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8138404], dtype=float32), 'logistic': array([0.14017461], dtype=float32), 'probabilities': array([0.85982543, 0.14017461], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.3677834], dtype=float32), 'logistic': array([0.2029782], dtype=float32), 'probabilities': array([0.7970218, 0.2029782], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9812416], dtype=float32), 'logistic': array([0.12118654], dtype=float32), 'probabilities': array([0.8788134 , 0.12118655], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8329594], dtype=float32), 'logistic': array([0.05556888], dtype=float32), 'probabilities': array([0.9444311 , 0.05556888], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.909467], dtype=float32), 'logistic': array([0.94831246], dtype=float32), 'probabilities': array([0.05168756, 0.94831246], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6777656], dtype=float32), 'logistic': array([0.06429818], dtype=float32), 'probabilities': array([0.93570185, 0.06429818], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4901235], dtype=float32), 'logistic': array([0.07655346], dtype=float32), 'probabilities': array([0.9234466 , 0.07655347], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9858353], dtype=float32), 'logistic': array([0.0480699], dtype=float32), 'probabilities': array([0.9519301 , 0.04806991], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.7305622], dtype=float32), 'logistic': array([0.02341781], dtype=float32), 'probabilities': array([0.9765822 , 0.02341781], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.10355384], dtype=float32), 'logistic': array([0.47413462], dtype=float32), 'probabilities': array([0.5258653 , 0.47413462], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.12644523], dtype=float32), 'logistic': array([0.4684308], dtype=float32), 'probabilities': array([0.53156924, 0.46843073], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.1059955], dtype=float32), 'logistic': array([0.75138175], dtype=float32), 'probabilities': array([0.2486182 , 0.75138175], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9293168], dtype=float32), 'logistic': array([0.05072322], dtype=float32), 'probabilities': array([0.9492768 , 0.05072321], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.5827293], dtype=float32), 'logistic': array([0.35830483], dtype=float32), 'probabilities': array([0.6416952 , 0.35830483], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6044114], dtype=float32), 'logistic': array([0.06885505], dtype=float32), 'probabilities': array([0.9311449 , 0.06885505], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4262745], dtype=float32), 'logistic': array([0.08119095], dtype=float32), 'probabilities': array([0.91880906, 0.08119094], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5261686], dtype=float32), 'logistic': array([0.07404391], dtype=float32), 'probabilities': array([0.925956 , 0.07404391], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1543393], dtype=float32), 'logistic': array([0.8960736], dtype=float32), 'probabilities': array([0.10392642, 0.8960736 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.33856195], dtype=float32), 'logistic': array([0.41615885], dtype=float32), 'probabilities': array([0.5838412 , 0.41615885], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7633388], dtype=float32), 'logistic': array([0.05933773], dtype=float32), 'probabilities': array([0.94066226, 0.05933773], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.43802434], dtype=float32), 'logistic': array([0.60778815], dtype=float32), 'probabilities': array([0.39221182, 0.60778815], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.6605018], dtype=float32), 'logistic': array([0.8403054], dtype=float32), 'probabilities': array([0.15969464, 0.8403054 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4654174], dtype=float32), 'logistic': array([0.07831839], dtype=float32), 'probabilities': array([0.92168164, 0.07831839], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6012243], dtype=float32), 'logistic': array([0.16781057], dtype=float32), 'probabilities': array([0.83218944, 0.16781057], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.62694246], dtype=float32), 'logistic': array([0.6517959], dtype=float32), 'probabilities': array([0.34820414, 0.6517959 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8446095], dtype=float32), 'logistic': array([0.13650705], dtype=float32), 'probabilities': array([0.86349297, 0.13650703], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.8798323], dtype=float32), 'logistic': array([0.29321253], dtype=float32), 'probabilities': array([0.70678747, 0.29321253], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0991914], dtype=float32), 'logistic': array([0.04314062], dtype=float32), 'probabilities': array([0.95685935, 0.04314062], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3084338], dtype=float32), 'logistic': array([0.09042689], dtype=float32), 'probabilities': array([0.9095731 , 0.09042688], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7597777], dtype=float32), 'logistic': array([0.14681819], dtype=float32), 'probabilities': array([0.8531818 , 0.14681819], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7622488], dtype=float32), 'logistic': array([0.0593986], dtype=float32), 'probabilities': array([0.9406014 , 0.05939861], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3901472], dtype=float32), 'logistic': array([0.03260481], dtype=float32), 'probabilities': array([0.96739525, 0.03260481], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.98420286], dtype=float32), 'logistic': array([0.7279414], dtype=float32), 'probabilities': array([0.27205864, 0.7279414 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.60459214], dtype=float32), 'logistic': array([0.3532938], dtype=float32), 'probabilities': array([0.6467062 , 0.35329378], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.01147], dtype=float32), 'logistic': array([0.1180039], dtype=float32), 'probabilities': array([0.88199615, 0.1180039 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9486164], dtype=float32), 'logistic': array([0.12470432], dtype=float32), 'probabilities': array([0.8752957, 0.1247043], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.342761], dtype=float32), 'logistic': array([0.08764288], dtype=float32), 'probabilities': array([0.9123571 , 0.08764288], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.65887266], dtype=float32), 'logistic': array([0.6590071], dtype=float32), 'probabilities': array([0.34099287, 0.6590071 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1499498], dtype=float32), 'logistic': array([0.8956641], dtype=float32), 'probabilities': array([0.10433591, 0.8956641 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.4248641], dtype=float32), 'logistic': array([0.60464656], dtype=float32), 'probabilities': array([0.3953534 , 0.60464656], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4196322], dtype=float32), 'logistic': array([0.08168784], dtype=float32), 'probabilities': array([0.91831213, 0.08168785], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.748317], dtype=float32), 'logistic': array([0.06018177], dtype=float32), 'probabilities': array([0.93981826, 0.06018177], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8463914], dtype=float32), 'logistic': array([0.05486815], dtype=float32), 'probabilities': array([0.9451319 , 0.05486815], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.373101], dtype=float32), 'logistic': array([0.08524701], dtype=float32), 'probabilities': array([0.914753 , 0.08524701], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3426602], dtype=float32), 'logistic': array([0.08765095], dtype=float32), 'probabilities': array([0.91234905, 0.08765095], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.799353], dtype=float32), 'logistic': array([0.9781049], dtype=float32), 'probabilities': array([0.02189513, 0.9781049 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.58036435], dtype=float32), 'logistic': array([0.35884875], dtype=float32), 'probabilities': array([0.64115125, 0.35884875], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1147056], dtype=float32), 'logistic': array([0.8923243], dtype=float32), 'probabilities': array([0.10767572, 0.8923243 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5261686], dtype=float32), 'logistic': array([0.07404391], dtype=float32), 'probabilities': array([0.925956 , 0.07404391], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6065052], dtype=float32), 'logistic': array([0.06872093], dtype=float32), 'probabilities': array([0.93127906, 0.06872093], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.501496], dtype=float32), 'logistic': array([0.62281084], dtype=float32), 'probabilities': array([0.37718916, 0.62281084], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.7377956], dtype=float32), 'logistic': array([0.3234864], dtype=float32), 'probabilities': array([0.6765136 , 0.32348636], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.37305564], dtype=float32), 'logistic': array([0.40780288], dtype=float32), 'probabilities': array([0.5921971 , 0.40780288], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3642364], dtype=float32), 'logistic': array([0.08594083], dtype=float32), 'probabilities': array([0.91405916, 0.08594082], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2840674], dtype=float32), 'logistic': array([0.09245113], dtype=float32), 'probabilities': array([0.9075489 , 0.09245112], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.5848625], dtype=float32), 'logistic': array([0.929881], dtype=float32), 'probabilities': array([0.07011902, 0.929881 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7057766], dtype=float32), 'logistic': array([0.15371232], dtype=float32), 'probabilities': array([0.84628767, 0.15371232], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.7495186], dtype=float32), 'logistic': array([0.93988615], dtype=float32), 'probabilities': array([0.06011384, 0.93988615], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2857865], dtype=float32), 'logistic': array([0.21656686], dtype=float32), 'probabilities': array([0.78343314, 0.21656683], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8839439], dtype=float32), 'logistic': array([0.13193652], dtype=float32), 'probabilities': array([0.8680635 , 0.13193652], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7959197], dtype=float32), 'logistic': array([0.05754507], dtype=float32), 'probabilities': array([0.942455 , 0.05754507], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9527406], dtype=float32), 'logistic': array([0.12425485], dtype=float32), 'probabilities': array([0.8757452 , 0.12425484], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.024848], dtype=float32), 'logistic': array([0.9536842], dtype=float32), 'probabilities': array([0.04631587, 0.9536842 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.3218334], dtype=float32), 'logistic': array([0.91066915], dtype=float32), 'probabilities': array([0.08933079, 0.91066915], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4524894], dtype=float32), 'logistic': array([0.0792567], dtype=float32), 'probabilities': array([0.92074335, 0.0792567 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9719923], dtype=float32), 'logistic': array([0.04870733], dtype=float32), 'probabilities': array([0.95129263, 0.04870733], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.2123423], dtype=float32), 'logistic': array([0.90135235], dtype=float32), 'probabilities': array([0.09864761, 0.90135235], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8084958], dtype=float32), 'logistic': array([0.05686681], dtype=float32), 'probabilities': array([0.94313323, 0.05686681], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.20995384], dtype=float32), 'logistic': array([0.5522965], dtype=float32), 'probabilities': array([0.44770348, 0.5522965 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.0149048], dtype=float32), 'logistic': array([0.4962739], dtype=float32), 'probabilities': array([0.5037261 , 0.49627388], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.5792875], dtype=float32), 'logistic': array([0.17089641], dtype=float32), 'probabilities': array([0.8291036, 0.1708964], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.47344178], dtype=float32), 'logistic': array([0.61619806], dtype=float32), 'probabilities': array([0.38380194, 0.61619806], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.1255076], dtype=float32), 'logistic': array([0.9579327], dtype=float32), 'probabilities': array([0.04206727, 0.9579327 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2310938], dtype=float32), 'logistic': array([0.22599006], dtype=float32), 'probabilities': array([0.77400994, 0.22599004], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.9486164], dtype=float32), 'logistic': array([0.12470432], dtype=float32), 'probabilities': array([0.8752957, 0.1247043], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6758802], dtype=float32), 'logistic': array([0.15764177], dtype=float32), 'probabilities': array([0.8423583 , 0.15764178], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6758802], dtype=float32), 'logistic': array([0.15764177], dtype=float32), 'probabilities': array([0.8423583 , 0.15764178], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.1053352], dtype=float32), 'logistic': array([0.89142066], dtype=float32), 'probabilities': array([0.10857935, 0.89142066], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.2504556], dtype=float32), 'logistic': array([0.22262126], dtype=float32), 'probabilities': array([0.7773787 , 0.22262126], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([4.121924], dtype=float32), 'logistic': array([0.9840454], dtype=float32), 'probabilities': array([0.01595462, 0.9840454 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4366162], dtype=float32), 'logistic': array([0.0804228], dtype=float32), 'probabilities': array([0.91957724, 0.08042281], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0104177], dtype=float32), 'logistic': array([0.11811347], dtype=float32), 'probabilities': array([0.88188654, 0.11811347], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.4168284], dtype=float32), 'logistic': array([0.9181016], dtype=float32), 'probabilities': array([0.08189841, 0.9181016 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.54403615], dtype=float32), 'logistic': array([0.36724913], dtype=float32), 'probabilities': array([0.6327508 , 0.36724913], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9636338], dtype=float32), 'logistic': array([0.04909608], dtype=float32), 'probabilities': array([0.9509039 , 0.04909608], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5519195], dtype=float32), 'logistic': array([0.07229764], dtype=float32), 'probabilities': array([0.9277024 , 0.07229764], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.46189648], dtype=float32), 'logistic': array([0.61346394], dtype=float32), 'probabilities': array([0.386536 , 0.61346394], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7621477], dtype=float32), 'logistic': array([0.05940425], dtype=float32), 'probabilities': array([0.94059575, 0.05940425], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.0126083], dtype=float32), 'logistic': array([0.04685951], dtype=float32), 'probabilities': array([0.95314056, 0.04685951], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7296622], dtype=float32), 'logistic': array([0.06124558], dtype=float32), 'probabilities': array([0.9387544 , 0.06124558], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3107316], dtype=float32), 'logistic': array([0.03520486], dtype=float32), 'probabilities': array([0.9647951 , 0.03520486], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6646388], dtype=float32), 'logistic': array([0.15914027], dtype=float32), 'probabilities': array([0.8408597 , 0.15914027], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.7685025], dtype=float32), 'logistic': array([0.85427135], dtype=float32), 'probabilities': array([0.14572866, 0.85427135], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.43624622], dtype=float32), 'logistic': array([0.3926358], dtype=float32), 'probabilities': array([0.6073642, 0.3926358], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2930777], dtype=float32), 'logistic': array([0.09169789], dtype=float32), 'probabilities': array([0.9083021 , 0.09169789], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.9271967], dtype=float32), 'logistic': array([0.05082539], dtype=float32), 'probabilities': array([0.94917464, 0.05082539], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.6008396], dtype=float32), 'logistic': array([0.02657527], dtype=float32), 'probabilities': array([0.9734247 , 0.02657527], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.0633711], dtype=float32), 'logistic': array([0.74333423], dtype=float32), 'probabilities': array([0.25666577, 0.74333423], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3428822], dtype=float32), 'logistic': array([0.08763321], dtype=float32), 'probabilities': array([0.9123668 , 0.08763321], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0574672], dtype=float32), 'logistic': array([0.11330004], dtype=float32), 'probabilities': array([0.8867 , 0.11330003], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.216966], dtype=float32), 'logistic': array([0.7715292], dtype=float32), 'probabilities': array([0.2284708, 0.7715292], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.9985375], dtype=float32), 'logistic': array([0.95250803], dtype=float32), 'probabilities': array([0.04749199, 0.95250803], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.6501275], dtype=float32), 'logistic': array([0.34296083], dtype=float32), 'probabilities': array([0.65703917, 0.3429608 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.743388], dtype=float32), 'logistic': array([0.06046116], dtype=float32), 'probabilities': array([0.9395389 , 0.06046116], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6399152], dtype=float32), 'logistic': array([0.0666133], dtype=float32), 'probabilities': array([0.93338674, 0.06661331], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.6247033], dtype=float32), 'logistic': array([0.16455725], dtype=float32), 'probabilities': array([0.8354428 , 0.16455725], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.45485705], dtype=float32), 'logistic': array([0.38820657], dtype=float32), 'probabilities': array([0.6117934 , 0.38820657], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.3192466], dtype=float32), 'logistic': array([0.21094365], dtype=float32), 'probabilities': array([0.78905636, 0.21094365], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.37605423], dtype=float32), 'logistic': array([0.40707892], dtype=float32), 'probabilities': array([0.5929211, 0.4070789], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.4793921], dtype=float32), 'logistic': array([0.38239568], dtype=float32), 'probabilities': array([0.6176043 , 0.38239568], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6358168], dtype=float32), 'logistic': array([0.06686858], dtype=float32), 'probabilities': array([0.93313146, 0.06686859], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.34706], dtype=float32), 'logistic': array([0.08729974], dtype=float32), 'probabilities': array([0.91270024, 0.08729975], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.4827917], dtype=float32), 'logistic': array([0.81499386], dtype=float32), 'probabilities': array([0.18500613, 0.81499386], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.10792582], dtype=float32), 'logistic': array([0.5269553], dtype=float32), 'probabilities': array([0.47304472, 0.5269553 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0841262], dtype=float32), 'logistic': array([0.11064926], dtype=float32), 'probabilities': array([0.8893508 , 0.11064927], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.48042685], dtype=float32), 'logistic': array([0.6178487], dtype=float32), 'probabilities': array([0.38215134, 0.6178487 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.137316], dtype=float32), 'logistic': array([0.10552245], dtype=float32), 'probabilities': array([0.89447755, 0.10552246], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2999605], dtype=float32), 'logistic': array([0.7858284], dtype=float32), 'probabilities': array([0.21417166, 0.7858284 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.38928968], dtype=float32), 'logistic': array([0.5961117], dtype=float32), 'probabilities': array([0.40388831, 0.5961117 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8880947], dtype=float32), 'logistic': array([0.05274523], dtype=float32), 'probabilities': array([0.9472548 , 0.05274523], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.4113041], dtype=float32), 'logistic': array([0.19602844], dtype=float32), 'probabilities': array([0.8039716 , 0.19602844], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.34706], dtype=float32), 'logistic': array([0.08729974], dtype=float32), 'probabilities': array([0.91270024, 0.08729975], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.9498808], dtype=float32), 'logistic': array([0.95025784], dtype=float32), 'probabilities': array([0.04974214, 0.95025784], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6778057], dtype=float32), 'logistic': array([0.06429576], dtype=float32), 'probabilities': array([0.93570423, 0.06429576], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.1443386], dtype=float32), 'logistic': array([0.04131493], dtype=float32), 'probabilities': array([0.95868504, 0.04131493], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.1913788], dtype=float32), 'logistic': array([0.10052735], dtype=float32), 'probabilities': array([0.89947265, 0.10052735], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.8657877], dtype=float32), 'logistic': array([0.05387095], dtype=float32), 'probabilities': array([0.946129 , 0.05387094], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.84821314], dtype=float32), 'logistic': array([0.29980782], dtype=float32), 'probabilities': array([0.70019215, 0.29980782], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.43984145], dtype=float32), 'logistic': array([0.60822123], dtype=float32), 'probabilities': array([0.39177877, 0.60822123], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.87488425], dtype=float32), 'logistic': array([0.294239], dtype=float32), 'probabilities': array([0.70576096, 0.294239 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.1600134], dtype=float32), 'logistic': array([0.04069853], dtype=float32), 'probabilities': array([0.9593015 , 0.04069853], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.8453041], dtype=float32), 'logistic': array([0.86357486], dtype=float32), 'probabilities': array([0.1364252 , 0.86357486], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7159994], dtype=float32), 'logistic': array([0.06203584], dtype=float32), 'probabilities': array([0.93796414, 0.06203584], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.027527], dtype=float32), 'logistic': array([0.11634292], dtype=float32), 'probabilities': array([0.8836571 , 0.11634292], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4939191], dtype=float32), 'logistic': array([0.07628557], dtype=float32), 'probabilities': array([0.9237144 , 0.07628556], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.5261686], dtype=float32), 'logistic': array([0.07404391], dtype=float32), 'probabilities': array([0.925956 , 0.07404391], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4900424], dtype=float32), 'logistic': array([0.0765592], dtype=float32), 'probabilities': array([0.92344075, 0.07655919], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.6892314], dtype=float32), 'logistic': array([0.84412307], dtype=float32), 'probabilities': array([0.15587695, 0.84412307], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.0002286], dtype=float32), 'logistic': array([0.11917893], dtype=float32), 'probabilities': array([0.8808211 , 0.11917892], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.552121], dtype=float32), 'logistic': array([0.07228412], dtype=float32), 'probabilities': array([0.92771584, 0.07228412], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.340217], dtype=float32), 'logistic': array([0.08784653], dtype=float32), 'probabilities': array([0.91215354, 0.08784652], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.7466123], dtype=float32), 'logistic': array([0.9769464], dtype=float32), 'probabilities': array([0.02305354, 0.9769464 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4643068], dtype=float32), 'logistic': array([0.07839859], dtype=float32), 'probabilities': array([0.9216014 , 0.07839859], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.760916], dtype=float32), 'logistic': array([0.0594731], dtype=float32), 'probabilities': array([0.94052684, 0.0594731 ], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.5058585], dtype=float32), 'logistic': array([0.37616488], dtype=float32), 'probabilities': array([0.62383515, 0.37616488], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.394353], dtype=float32), 'logistic': array([0.19871373], dtype=float32), 'probabilities': array([0.8012862 , 0.19871372], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.429804], dtype=float32), 'logistic': array([0.919072], dtype=float32), 'probabilities': array([0.08092804, 0.919072 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2767317], dtype=float32), 'logistic': array([0.09306845], dtype=float32), 'probabilities': array([0.9069315 , 0.09306846], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.350621], dtype=float32), 'logistic': array([0.03387484], dtype=float32), 'probabilities': array([0.9661252 , 0.03387484], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.3676736], dtype=float32), 'logistic': array([0.03332116], dtype=float32), 'probabilities': array([0.96667886, 0.03332116], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.2906396], dtype=float32), 'logistic': array([0.09190115], dtype=float32), 'probabilities': array([0.9080989 , 0.09190115], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.7021495], dtype=float32), 'logistic': array([0.15418473], dtype=float32), 'probabilities': array([0.84581524, 0.15418473], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.891106], dtype=float32), 'logistic': array([0.947405], dtype=float32), 'probabilities': array([0.05259499, 0.947405 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4648888], dtype=float32), 'logistic': array([0.07835655], dtype=float32), 'probabilities': array([0.92164344, 0.07835656], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([3.018937], dtype=float32), 'logistic': array([0.9534223], dtype=float32), 'probabilities': array([0.04657765, 0.9534223 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.74305326], dtype=float32), 'logistic': array([0.67766315], dtype=float32), 'probabilities': array([0.32233682, 0.67766315], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.9401757], dtype=float32), 'logistic': array([0.87437147], dtype=float32), 'probabilities': array([0.12562856, 0.87437147], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.0429635], dtype=float32), 'logistic': array([0.2605786], dtype=float32), 'probabilities': array([0.7394214, 0.2605786], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.2021129], dtype=float32), 'logistic': array([0.7689004], dtype=float32), 'probabilities': array([0.23109956, 0.7689004 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.4534457], dtype=float32), 'logistic': array([0.03066627], dtype=float32), 'probabilities': array([0.9693337 , 0.03066627], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.717829], dtype=float32), 'logistic': array([0.15215102], dtype=float32), 'probabilities': array([0.847849 , 0.15215102], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.5187134], dtype=float32), 'logistic': array([0.6268469], dtype=float32), 'probabilities': array([0.37315315, 0.6268469 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.0342655], dtype=float32), 'logistic': array([0.88434803], dtype=float32), 'probabilities': array([0.11565194, 0.88434803], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-0.83103323], dtype=float32), 'logistic': array([0.30342665], dtype=float32), 'probabilities': array([0.6965734 , 0.30342665], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.4835986], dtype=float32), 'logistic': array([0.18488449], dtype=float32), 'probabilities': array([0.8151155 , 0.18488449], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6777656], dtype=float32), 'logistic': array([0.06429818], dtype=float32), 'probabilities': array([0.93570185, 0.06429818], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.9513098], dtype=float32), 'logistic': array([0.87558943], dtype=float32), 'probabilities': array([0.12441061, 0.87558943], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.0337292], dtype=float32), 'logistic': array([0.26236176], dtype=float32), 'probabilities': array([0.73763824, 0.26236176], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-3.5533388], dtype=float32), 'logistic': array([0.02783209], dtype=float32), 'probabilities': array([0.97216785, 0.02783209], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.041459], dtype=float32), 'logistic': array([0.8850817], dtype=float32), 'probabilities': array([0.11491823, 0.8850817 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([1.5367038], dtype=float32), 'logistic': array([0.8229851], dtype=float32), 'probabilities': array([0.17701496, 0.8229851 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.4166243], dtype=float32), 'logistic': array([0.08191376], dtype=float32), 'probabilities': array([0.91808623, 0.08191376], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.3841245], dtype=float32), 'logistic': array([0.08439132], dtype=float32), 'probabilities': array([0.9156087 , 0.08439132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.7616632], dtype=float32), 'logistic': array([0.05943133], dtype=float32), 'probabilities': array([0.9405686 , 0.05943132], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.49434847], dtype=float32), 'logistic': array([0.6211303], dtype=float32), 'probabilities': array([0.3788697, 0.6211303], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-1.8977379], dtype=float32), 'logistic': array([0.13036472], dtype=float32), 'probabilities': array([0.8696353, 0.1303647], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6399152], dtype=float32), 'logistic': array([0.0666133], dtype=float32), 'probabilities': array([0.93338674, 0.06661331], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([2.666312], dtype=float32), 'logistic': array([0.9350093], dtype=float32), 'probabilities': array([0.06499072, 0.9350093 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([0.24823135], dtype=float32), 'logistic': array([0.5617411], dtype=float32), 'probabilities': array([0.43825886, 0.5617411 ], dtype=float32), 'class_ids': array([1]), 'classes': array([b'1'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}, {'logits': array([-2.6578376], dtype=float32), 'logistic': array([0.06550758], dtype=float32), 'probabilities': array([0.93449247, 0.06550758], dtype=float32), 'class_ids': array([0]), 'classes': array([b'0'], dtype=object), 'all_class_ids': array([0, 1], dtype=int32), 'all_classes': array([b'0', b'1'], dtype=object)}]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "QTKZMrZ0jXEJ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "e1f5f295-7c66-4c19-a90b-cd094da34861"
},
"source": [
"print(\"Probability that in Titanic during a shipwreck, \")\n",
"print(\"A random person will survive : \", pred[0]['probabilities'][1])\n",
"print(\"A random person will die : \", pred[0]['probabilities'][0])"
],
"execution_count": 99,
"outputs": [
{
"output_type": "stream",
"text": [
"Probability that in Titanic during a shipwreck, \n",
"A random person will survive : 0.08350867\n",
"A random person will die : 0.9164914\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IpRfDFpCo16w",
"colab_type": "text"
},
"source": [
"Let's see a more detailed prediction about few people in titanic based on their details."
]
},
{
"cell_type": "code",
"metadata": {
"id": "wT0alX7yj5dm",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "3eb716d0-95ce-4d1a-f212-72aac8fd8332"
},
"source": [
"for i in range(2, 20, 2):\n",
" print(\"-----\\nDetails of a person: \\n\", df_test.loc[i])\n",
" print(\"Probabiliy of survival: \", pred[i]['probabilities'][1])\n",
" print(\"Probabiliy of death: \", pred[i]['probabilities'][0])\n",
" print()"
],
"execution_count": 100,
"outputs": [
{
"output_type": "stream",
"text": [
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 58\n",
"n_siblings_spouses 0\n",
"parch 0\n",
"fare 26.55\n",
"class First\n",
"deck C\n",
"embark_town Southampton\n",
"alone y\n",
"Name: 2, dtype: object\n",
"Probabiliy of survival: 0.5778602\n",
"Probabiliy of death: 0.42213982\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex male\n",
"age 34\n",
"n_siblings_spouses 0\n",
"parch 0\n",
"fare 13\n",
"class Second\n",
"deck D\n",
"embark_town Southampton\n",
"alone y\n",
"Name: 4, dtype: object\n",
"Probabiliy of survival: 0.07680218\n",
"Probabiliy of death: 0.9231978\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 8\n",
"n_siblings_spouses 3\n",
"parch 1\n",
"fare 21.075\n",
"class Third\n",
"deck unknown\n",
"embark_town Southampton\n",
"alone n\n",
"Name: 6, dtype: object\n",
"Probabiliy of survival: 0.51535875\n",
"Probabiliy of death: 0.4846412\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 18\n",
"n_siblings_spouses 2\n",
"parch 0\n",
"fare 18\n",
"class Third\n",
"deck unknown\n",
"embark_town Southampton\n",
"alone n\n",
"Name: 8, dtype: object\n",
"Probabiliy of survival: 0.83662623\n",
"Probabiliy of death: 0.16337374\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 28\n",
"n_siblings_spouses 0\n",
"parch 0\n",
"fare 7.75\n",
"class Third\n",
"deck unknown\n",
"embark_town Queenstown\n",
"alone y\n",
"Name: 10, dtype: object\n",
"Probabiliy of survival: 0.03349888\n",
"Probabiliy of death: 0.9665011\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 5\n",
"n_siblings_spouses 1\n",
"parch 2\n",
"fare 27.75\n",
"class Second\n",
"deck unknown\n",
"embark_town Southampton\n",
"alone n\n",
"Name: 12, dtype: object\n",
"Probabiliy of survival: 0.14107606\n",
"Probabiliy of death: 0.8589239\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 29\n",
"n_siblings_spouses 0\n",
"parch 0\n",
"fare 10.5\n",
"class Second\n",
"deck F\n",
"embark_town Southampton\n",
"alone y\n",
"Name: 14, dtype: object\n",
"Probabiliy of survival: 0.56723243\n",
"Probabiliy of death: 0.43276757\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex male\n",
"age 26\n",
"n_siblings_spouses 1\n",
"parch 0\n",
"fare 14.4542\n",
"class Third\n",
"deck unknown\n",
"embark_town Cherbourg\n",
"alone n\n",
"Name: 16, dtype: object\n",
"Probabiliy of survival: 0.10752523\n",
"Probabiliy of death: 0.8924748\n",
"\n",
"-----\n",
"Details of a person: \n",
" sex female\n",
"age 33\n",
"n_siblings_spouses 3\n",
"parch 0\n",
"fare 15.85\n",
"class Third\n",
"deck unknown\n",
"embark_town Southampton\n",
"alone n\n",
"Name: 18, dtype: object\n",
"Probabiliy of survival: 0.41871348\n",
"Probabiliy of death: 0.58128655\n",
"\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "KWbN7IIdfs_b",
"colab_type": "text"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "jdPm5S14ptcW",
"colab_type": "text"
},
"source": [
"## Basic Image Classification with Tensorflow and keras"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "itoR1B6_qIXs",
"colab_type": "text"
},
"source": [
"### Import Packages"
]
},
{
"cell_type": "code",
"metadata": {
"id": "XPaBsdJMpsm_",
"colab_type": "code",
"colab": {}
},
"source": [
"# TensorFlow and tf.keras\n",
"import tensorflow as tf\n",
"from tensorflow import keras\n",
"\n",
"# Helper libraries\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"from IPython.display import clear_output\n",
"\n",
"# Hide warnings\n",
"import warnings\n",
"warnings.filterwarnings('ignore')"
],
"execution_count": 101,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "DyldeMVrqMIs",
"colab_type": "text"
},
"source": [
"### Import the Fashion MNIST Dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "_Fi0XZzLqMnP",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 150
},
"outputId": "e33fd5e5-747e-4247-98c5-c54ddebbc33f"
},
"source": [
"fashion_mnist = keras.datasets.fashion_mnist\n",
"\n",
"(train_images, train_labels), (test_images, test_labels) = fashion_mnist.load_data()"
],
"execution_count": 102,
"outputs": [
{
"output_type": "stream",
"text": [
"Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-labels-idx1-ubyte.gz\n",
"32768/29515 [=================================] - 0s 0us/step\n",
"Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/train-images-idx3-ubyte.gz\n",
"26427392/26421880 [==============================] - 0s 0us/step\n",
"Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-labels-idx1-ubyte.gz\n",
"8192/5148 [===============================================] - 0s 0us/step\n",
"Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/t10k-images-idx3-ubyte.gz\n",
"4423680/4422102 [==============================] - 0s 0us/step\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J2HtkM88s_z2",
"colab_type": "text"
},
"source": [
"### About the Fashion MNIST dataset\n",
"\n",
"The images are 28x28 NumPy arrays, with pixel values ranging from 0 to 255. The labels are an array of integers, ranging from 0 to 9. These correspond to the class of clothing the image represents:\n",
"\n",
"| label | class |\n",
"|---|---|\n",
"| 0 | T-shirt/Top |\n",
"| 1 | Trouser |\n",
"| 2 | Pullover |\n",
"| 3 | Dress |\n",
"| 4 | Coat |\n",
"| 5 | Sandal |\n",
"| 6 | Shirt |\n",
"| 7 | Sneaker |\n",
"| 8 | Bag |\n",
"| 9 | Ankle Boot |\n",
"\n",
"Each image is mapped to a single label. Since the class names are not included with the dataset, store them here to use later when plotting the images:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "rnPYUtVDrU1O",
"colab_type": "code",
"colab": {}
},
"source": [
"class_names = ['T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',\n",
" 'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle boot']"
],
"execution_count": 103,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "7uWnRWxOrWfR",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "de51656d-1670-4325-8b0d-0217415cfe7f"
},
"source": [
"train_images.shape"
],
"execution_count": 104,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(60000, 28, 28)"
]
},
"metadata": {
"tags": []
},
"execution_count": 104
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "0TkBPuASrdwO",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "1965ba87-9ca8-499f-c72e-c3c4302f3802"
},
"source": [
"test_images.shape"
],
"execution_count": 105,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"(10000, 28, 28)"
]
},
"metadata": {
"tags": []
},
"execution_count": 105
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7YlSKQPtpvbP",
"colab_type": "text"
},
"source": [
"### Preprocess the data"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "mqwJrnAtvM45",
"colab_type": "text"
},
"source": [
"The data must be preprocessed before training the network. If you inspect the first image in the training set, you will see that the pixel values fall in the range of 0 to 255:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "KHrlSeqkummm",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 252
},
"outputId": "f20cb948-c209-4709-934b-5fdd1994237f"
},
"source": [
"plt.figure()\n",
"plt.imshow(train_images[0], cmap='binary')\n",
"plt.xticks([])\n",
"plt.yticks([])\n",
"plt.grid(False)\n",
"plt.grid(False)\n",
"plt.show()"
],
"execution_count": 106,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAOsAAADrCAYAAACICmHVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAK0UlEQVR4nO3dzW/M6x/G8bseSqccj1VKVTwkmrQiMqS6bEJUYuEvsLKysCJi4S+wtyARG0klGisREhIaGkY0FiIlFE1plSjV1lN7tueXn+/1MTN1uJz3a3udezrT6ZWvnE/u+66YmppKAH5/M371GwDwYygrYIKyAiYoK2CCsgImKCtgYlYx//HSpUun1qxZ85PeCoC+vr40PDxc8b2sqLKuWbMmFQqF6XlXAP5PPp/PzPhnMGCCsgImKCtggrICJigrYIKyAiYoK2CCsgImKCtggrICJigrYIKyAiYoK2CCsgImKCtggrICJigrYIKyAiYoK2CCsgImijowDf++6OKwiorvHoT3wz58+CDzrq6uzKy9vb2snx19tm/fvmVms2b92j/dci50K/U748kKmKCsgAnKCpigrIAJygqYoKyACcoKmGDO+pubnJyU+cyZM2X++PFjmZ86dUrmVVVVmVl1dbVcO3fuXJlv27ZN5uXMUqM5aPR7jdaX897U/FjhyQqYoKyACcoKmKCsgAnKCpigrIAJygqYYM76m4tmctGc9erVqzK/cuWKzOvr6zOzT58+ybVjY2Myv3z5ssz379+fmdXW1sq10Z7R6PcWGR0dzcxmzNDPwFwuV9LP5MkKmKCsgAnKCpigrIAJygqYoKyACcoKmGDO+purrKwsa/2dO3dk3tfXJ3O17zPaE7pz506Z37t3T+aHDx/OzPL5vFzb3Nws88bGRpnfvn1b5ur32traKtdu3749M1NzdZ6sgAnKCpigrIAJygqYoKyACcoKmGB08xtQx15GW72iLW6FQkHmf/31l8w/fvyYmfX29sq1Ub5161aZr1+/PjNTW9RSSunmzZsy7+zslHl01Kg6RvXkyZNyrRrHqW2FPFkBE5QVMEFZAROUFTBBWQETlBUwQVkBExXR1Xb/lM/np6K53X9RMb/DYkVz1paWFplHW+Ai6rNFx3nOmTOnrJ+troyMfi9btmyR+YYNG2QefbZLly5lZk+ePJFrBwYGMrN8Pp8KhcJ3PxxPVsAEZQVMUFbABGUFTFBWwARlBUxQVsAE+1mnQTTz+5kWLVok85cvX8q8qqpK5upaxy9fvsi10Z5TNUdNKaXx8fHMLPqdd3V1yTza7xrNzgcHBzOzXbt2ybWl4skKmKCsgAnKCpigrIAJygqYoKyACcoKmGDOak6dM5uSvkIwpfjaRjWHXb58uVy7ZMkSmUd7bWfMyH6WRHPQ6HOrGW70s1PS+137+/vl2lLxZAVMUFbABGUFTFBWwARlBUxQVsAEZQVMMGedBtHML5plqpldtCdUnUGbUnx2r7orNKWUPn/+XPJrV1dXy3xkZETmak4bzZfV+04ppXnz5sn8/fv3Mm9ubs7M1J22Kek7c7mfFfgDUFbABGUFTFBWwARlBUxQVsAEo5tpEB2LGW3XUqObjo4OuTY6arSmpkbm0VYx9d6iEcXz589lPnv2bJmrY1BnzdJ/utExqdHnHh4elvmBAwcys56eHrn269evmZkaA/JkBUxQVsAEZQVMUFbABGUFTFBWwARlBUwwZ50Gam6WUrwNTWlqapJ5tE0tmjeWMwMeGhqSa6MrHRcvXixz9XuNPlc0A46uyqyvr5f52bNnM7NDhw7JtS0tLZmZ2lbIkxUwQVkBE5QVMEFZAROUFTBBWQETlBUw8a/OWdVevXKvJoyOA1V7J6Pr/SLR3spytLe3yzw6UlNd2ZhSfGSnEu2VjebPExMTMi9nPh19J9F3Hv093r9/PzNbsGCBXFsqnqyACcoKmKCsgAnKCpigrIAJygqYoKyAiWkdEJazN/Jnzip/tuvXr8v8/PnzMu/q6srMcrmcXKuuRUxJn72bUnzmsfpeovcW/T1E703NYaP3HV03GYnmz+r1Ozs75do9e/aU9J54sgImKCtggrICJigrYIKyAiYoK2CCsgImpnW4qeao5Xr79q3MBwYGZN7b21vy2mhupl47pfhsX7VXN5oXvnnzRuZ1dXUyj872VefzDg4OyrXR5x4bG5N5a2trZvbhwwe59saNGzKP9rNGe1LV/uju7m65tlQ8WQETlBUwQVkBE5QVMEFZAROUFTAxraObW7duyfzYsWOZ2evXr+Xad+/eyTz6X/FqPLJw4UK5NhpJzZ8/X+bRCEMdoxodJarGGyml1NHRIfOtW7fK/P3795lZNPbp6+uTeUQd9zk6OirXrlq1SubRSCwaK6krJcv93Fl4sgImKCtggrICJigrYIKyAiYoK2CCsgImip6zquMlDx48KNeqrWjlXtFXztGT0ZGY0awzyiMjIyOZ2bNnz+TaI0eOyDx6bydOnJD5ihUrMrNoztrW1ibzdevWyfzRo0eZWbQ1UG1hSym+jjK6YlT9vS5btkyuLRVPVsAEZQVMUFbABGUFTFBWwARlBUxQVsBEUXPW4eHhdObMmcw8mgmuXbs2M1P7A1OKj56M5m5KNHNTc9CU4r2TK1eulPn4+HhmVltbK9fu27dP5hcuXJB5dP3g06dPM7PoO7t7967Mr127JnM104/2CEez8+hKx4ias0av/eLFi5LW8mQFTFBWwARlBUxQVsAEZQVMUFbABGUFTBQ1Z509e7bcqxfNG9WsNJqbrV69uuTXTklfXajOxk0ppcWLF8u8oaFB5tF7U/tCoz2j0ZnGe/fulXlzc7PM1Rm40Ww7+k6j85rVntToc1dWVso8moVG+6fVWc8qS0lfEarmwzxZAROUFTBBWQETlBUwQVkBE5QVMFH06EaNZ6L/3V1fX5+ZRdutoishozFATU1NSVlK8Ra6aDtWtH5iYiIzi642VNvIUkppyZIlMn/w4IHM582bl5lF47RFixbJXH3ulPT3Eh1dGx1FGq1X2xZTSunVq1eZ2YIFC+Tanp6ezExdNcmTFTBBWQETlBUwQVkBE5QVMEFZAROUFTBR1Jw1l8ulzZs3Z+bRdqzTp09nZnV1dXJtdD1gtJVMzSuj7VLRzE1tv0spnrOq9x6traiokHkul5O5utIxJT07j7apRe89mo2Xs6Uyeu0oj7bYqTmuOr41JX28rHpdnqyACcoKmKCsgAnKCpigrIAJygqYoKyAiYro2MR/yufzU4VCoeQfdvHixczs+PHjcu3Q0JDMoz2paq4W7cOdnJyUebSfNdpzquaR0fcTzVmjWWc0Y1Z59NrF/G0Vu14difsjotl49Deh9rNu2rRJrj137lxmls/nU6FQ+O6XypMVMEFZAROUFTBBWQETlBUwQVkBE5QVMFHUftaU9Mwxmk3t3r27pCyllK5evSrzo0ePylxdXTgyMiLXRvPCaI4azfTUGbbRz47mjdEcNrqmU+21VWcKpxT/XsoR7TeN9vFGs/MdO3bIvLGxMTNrbW2Va0vFkxUwQVkBE5QVMEFZAROUFTBBWQETlBUwUfScNZql/ixtbW0y7+7uLvm1Hz58KPPobtjoHtL+/n6ZNzQ0ZGbRPDE6Txl/Dp6sgAnKCpigrIAJygqYoKyACcoKmCh6dPMn2rhxY1l5pKmpqaz1QEo8WQEblBUwQVkBE5QVMEFZAROUFTBBWQETlBUwQVkBE5QVMEFZAROUFTBBWQETlBUwQVkBExXRlYL/8x9XVLxOKT37eW8H+M9rmJqaqvleUFRZAfw6/DMYMEFZAROUFTBBWQETlBUwQVkBE5QVMEFZAROUFTDxN/Yv91E2qyW8AAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "mTlTY3dZunAS",
"colab_type": "code",
"colab": {}
},
"source": [
"# scaling the dataset images between 0 to 1\n",
"# for feeding into neural network model\n",
"train_images = train_images / 255.0\n",
"test_images = test_images / 255.0"
],
"execution_count": 107,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "uvqr9Nx0upZe",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 588
},
"outputId": "32c6b9f1-6edf-4f53-b4b6-df5e739438fe"
},
"source": [
"# displaying first 25 images to check if class names labelling is wrong\n",
"plt.figure(figsize=(10,10))\n",
"for i in range(25):\n",
" plt.subplot(5,5,i+1)\n",
" plt.xticks([])\n",
" plt.yticks([])\n",
" plt.grid(False)\n",
" plt.imshow(train_images[i], cmap='binary')\n",
" plt.xlabel(class_names[train_labels[i]])\n",
"plt.show()"
],
"execution_count": 108,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAj0AAAI8CAYAAAAazRqkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydebxd0/n/P0tQEYSMMroSMTRE5hiCmIUoalZTfUv9Wi0dDNXSUq2qKlVTSxU1FkkNRYJIhEhlEBmEyEhEElcSEVKK/fvjnrvyWU/OXtn35p57z73783698sqzz1pnnX32WmuffZ/RJUkCIYQQQoimzkYNfQJCCCGEEPWBHnqEEEIIkQv00COEEEKIXKCHHiGEEELkAj30CCGEECIX6KFHCCGEELlg45p0btOmTVJRUVGiUxHFWLBgASorK11dj1suc/nf//7Xy++8846Xt9lmm6Df5ptv7mXnXFHZjrdixQovf+1rXwv6bbvttl5u1qxZTU+71kyePLkySZK2dT1uQ83nF198ERxXVlZ6uXXr1l7eZJNNNvizPv30Uy/zPAPherFrolQ0hb352WefeXn16tVB28qVK73Me4TnFQj3Ztr+A4CPP/7YyxtttPbv7VatWgX92rat8+2RiVLszXK5z5aS//3vf16ui31eF8TmskYPPRUVFZg0aVLdnJXIRP/+/Usybl3MJed4qu0PzaxZs7x83nnnefmEE04I+vXp08fLm266qZc33jhcwjNnzvTyiBEjvNytW7eg30UXXeTlrbfeuqanXWuccwtLMW5D7c1ly5YFx3fddZeXTz/9dC/zQ2ZtmTp1qpfffPPNoO3YY4/1cn3deMt5b2Zl/vz5Xh47dmzQ9thjj3mZH0xOO+20oF/fvn29zPPy6KOPBv2ee+45L7do0cLLp556atDvnHPOyXTudU0p9mYefjMXL17s5Y4dOzbgmawlNpcybwkhhBAiF9RI0yPyR0ybk6bdee2114Ljhx56yMv2rz9Wm7N6/dJLLw36LV++POMZr2XHHXf08uuvvx60XX311V5mLcShhx4a9PvJT37i5d12263G59AU4Xl6/PHHg7Z77rnHyw8++KCXrcmCtXWsmbEmFja/vPvuu14++uijg368jo4//vj4F8gZTz/9tJevv/76oK158+Ze/vzzz4O2zTbbzMsLFizw8kknnRT0W7p0qZfZlGO1sB06dPByy5YtvfzII48E/W644QYvH3TQQV6+8cYbIdI54IADvGxNi23atPHy7bff7uWspjfW5gDA/vvv7+U1a9Z4uWvXrkG/kSNHepm1ew2JND1CCCGEyAV66BFCCCFELtBDjxBCCCFygXx6RJRYVNaqVau8zJE61n+G/YK22GKLoI19Cjjs2IaRc2j0Rx995GUOl7Xvi537wIEDvcxhtuPHjw/6jRkzxsuDBw8O2u69997U8ZsyPIfsmwEAv/vd77z8m9/8xss22or9QNhvx0bSbbnlll5m/47DDz886Gd9gfLO3LlzvXz//fd72fqlsT/GV199FbRxWHmXLl28vNVWW6V+Lu85u4f5fezHZX1/9txzTy8vWrTIy+xfBwDXXXdd6nnkEZ4/Th0BAO+9956XeQ3Y+/Fxxx3nZb6/ffnll0E/9vfiPctpCYDy8eNhpOkRQgghRC7QQ48QQgghckGTMm+xGQVIN29YFdxLL73k5aFDh2Yan9V9Vj2bFXu+TH1lld0QjjnmGC9zNuX27dsH/fi7WDVpWjZk24+vFWeEtf3S3hODTWystgXCcx83blzQxokVd9lll0yf1dRg0xQQqrq///3ve/nPf/5z0I8zZMfMW/369fPyt7/9bS9zCDXQcFl8yxU2/cSuDZtEbJZr3pt8j9t+++2Dfmzi5DHsPcyulWJjA2GGXw6pnjFjRtDvySef9PKwYcOKjp0nOIEkJ50Ewnsmp/9YsmRJ0I/3KbspTJs2LejHrgg8XzZbdzkiTY8QQgghcoEeeoQQQgiRC5qUectGH7B6ds6cOV6+4447gn5s3mBvc2vq4IifmEmLzSr2nLgtNkbMbNNQTJ48OThmkxZn/LRFKBmOFgHCqIJYJAlfK742HGFi4Qyzth4TRwV17ty56OdY7GfxOsprJAlfRyCMGtluu+28bK8Pz/sHH3zgZZshltcVj23XWFZTZl4488wzvcxZmK2pi03R1uyfVsOMs2kD4fwxNsrLRlqmweNz0VPep4BMWpbu3bt7ecKECUEb/xba4stp8F60pn2uscX3bS4KXK5I0yOEEEKIXKCHHiGEEELkAj30CCGEECIXNCmfnlg49OjRo7387LPPBv042yiHVVr75KhRo7x89tlnezkWop0Wkg2EWWStv0hW+3d98sILLwTHfK04VNV+F/bPsfbk3//+917mKsw8J0BY5Zf7Wd8f9kNgnx6bsXfKlCle5urN1ueBwzHt9+KK8Xn16Ymt7w8//DC1jX11uMq93XPs+xPLtt0YUjzUJ+x/yBmOH3vssaDfoEGDvGz9pHguOBza+vTwnmE/SDuXvJc4zH3ZsmUp3yL0F+Fs32JdOG2GvS/y/mC/VTuXNjS9Guvfyj50PK+xbN3lgjQ9QgghhMgFeugRQgghRC5oUuYtq6pjJk6c6GWbzZVVgSwfcsghQb/XXnvNyxdddJGX+/fvH/Tjgm42U++rr75a9Jz22muvoF+1SrqcQtcfeeSR4JjNDXzdbNg3q7ltgUo2E7L50IbHn3XWWV7+y1/+4uWePXsG/djMxteuXbt2Qb8f/ehHXr7lllu8zKpaO54tnsdFNGfPnu3lHXfcEXkhlgWd14ddxxyKXJvPsuasWJqEvPPDH/7QyzfccEPQxmkFrGmX1zub22MmDJ4HOx63xUwiXFCYM+Q3BtNJQxJLvcH7j83+7CoAAH369PEyX2+bLsCaz6qx9/dyRJoeIYQQQuQCPfQIIYQQIhc0evNWTOXNUVqTJk3yslWTfvLJJ15mMwXLADBgwAAv77DDDl62kUHjx4/38vDhw4M2VjtyhMXtt98e9Ks21ZVThksuQAeEEVasPk0rLAiEqmvLoYce6uUtttgiaOPinn/4wx+8zEVPAeCJJ57wMqvTWW0LhNFbPCf2enPElo3e4u//yiuveDlP5i279nnuOeLDmrf4WnJbLLNymhkaWLdYZt7htc/r++WXXw76/fznP08dg01aHBVps6pzRnueS9uPIzfTzCO27cgjj0ztJ0LYVGWzafO+YrOz7cfuAmyCtPPFZize87F5LRek6RFCCCFELtBDjxBCCCFygR56hBBCCJELGoVPT20rKF922WVefv/991P7sR9HrBrtSy+95GX2EbK+RH379vVyjx49gjYe/6abbvLyvHnzgn7V2X5tFev6Zvr06V62IahpIcnWf4Nt+5zZ1TJz5kwv22vP88d+CHZtsI2a29jnxsK2cM78DMSzALMvw4svvujlM844I/WzmhqxaucsW1t/bfqxb4rtV06pHcoBG7JcjQ1R7tatm5fnz58ftLFPFt+HrG8b9+N5sX55XI09Npddu3Yteu4iDt+fbVqWnXfe2cs8X/b+aVN2VBPzEeL1EEsbUy5I0yOEEEKIXKCHHiGEEELkgkZh3qptMcFtttnGy2weYbMEEIbcsXrPhuOyWpBNNvb82AzG4etAqBZcunSplw877LCUb9GwXHPNNV62IaicsTUW9s3XzapJ2UzIBSqXL18e9ON54etmx+PP4syjNgPwQw895OUVK1Z42a4Nfp9t43OyGaTzgjVNcJgzm5xiZqtY0dK0vW/Nn6J28DzY+x2bLfgeaU3uvM94/8VMHbE5t9nTRTa4cK8lrUBoLMSc9541Y/Mx73P+zS1XpOkRQgghRC7QQ48QQgghcoEeeoQQQgiRCxqFT09tYd+SmH8B+2qwXbR169ZBPw4DZHu3DfuLpWLn97Fde9GiRcW/RAPD1d/ZlwYA5syZ42UuL2F9ejhs34a7Dho0yMt8PWw/Pub5syGWaSHONqSZS5Fw2QguSWI/y85zx44dvXz00Ucjj8R8Avia2/mM7cc02I/A+vTYtSnWwtfXzkOnTp28PG3atNT38fW2Y3AJEG6zpUH4Psu+P5WVlUE/W9G7GutXkhaWL8LrWxPYj4dl64PF157vi7bEUzkiTY8QQgghcoEeeoQQQgiRCxqFftCaFVjtymo3G3LJ2XVZPWtDKTnkkvtxSDYQmnDY9GXNOTyezUq6atUqL++2225etmaV6lDuhq6y/r3vfa+oDISh3m+//baXb7311qDfmDFjvGwzMvM12Hrrrb3M1xCoXfXeWKZfVv/yvPbq1Svod//999f4c5s6PO/WbMjXnNXjta2+zOYSNm9Y9T3vEzar1FbNnxcqKiq8bOeS9yDP+XbbbRf0Y1MHp52w4cvcj+/B9v4us9WGkzXNi+2Xtn9tP97P3GZ/M8sRaXqEEEIIkQv00COEEEKIXNAo9IhWtcZqWDZvcZZdIMzCzMXYbEQVj8FmpnfeeSfox9l/OUOpVcdyRJH9LI5U+P73v+/lqVOnBv2qVfm1LbZaH7D6euDAgV62kTWjR4/2sp1Lvo587W2kho0YqcZen7RCePw5QDiXbA7haDVRHJ5fO9e1VatXEzNlM9YU07JlSy/LpJUdzqAdy5KcFj0JpEdvWfMWFxy1rgiMNW2LmpP1d8P24/tuLPqV55nlZcuW1eg8GwJpeoQQQgiRC/TQI4QQQohcoIceIYQQQuSCRuHTY/070qr37rrrrsEx+xuwn421T7Itm22S1jeAw635nGxWYPZNsXbtLl26eJnDoS+88MKg3x577AGgvEIArf2XvzfPifXX4KrMsWsf8wdJC6WsLWm+Ihw2b4nZtevinBoL/F3tNamvz7U+WiKdNH84IPTbYL9HINzTserZvGf4PdafsX379l5m/55yusc1FWrr05MWih7z/WH/SK5aUK5I0yOEEEKIXKCHHiGEEELkgjozb7H6K1ZMkPuxWiyrCjbG0KFDg2POhszF7mIhkazitWY1Ds1MM7EB4fnGCi1ygT8OuS1XrAmH54/p3r17cMxF6LKaKrNmCs1KLAs3E5sHu5ZjIb5NmZhJKxbaXJfvic1FrMBmHoldD84Qz1mXgfCeyZmWLXzP5MzYnOkcSN/rdi5tqpBqlKk5OzHzVqyIctoYWdPGyLwlhBBCCFEm6KFHCCGEELmg1vrCWBROXashX3zxxeD40Ucf9fJLL73kZc4uCoRFQTnaw6rq+Hx5DPsdeQw2ddnxYtEIbFbhfsOHDw/6HXnkkaljlAtphV9ZLQ6EUXR83YDQRMbRYFbtmhZJkDWDb6xAJY+RV5NVTYit/bR5steV5ylrBFhM3c7HvMeUnTlu4mPTVM+ePYO2rl27epn3i72mS5cu9TKbsGxhUn4fm9U6dOgQ9HvvvfdSz1ekM3v2bC9b833W4r+xe2taP/795IoD5Yo0PUIIIYTIBXroEUIIIUQu0EOPEEIIIXJBrZ1vsvo+LF++PDhevHixl9kGya8DoY8L9wNCHxG2T1pfGg6z7Nixo5etTZp9Sdg+bStIs12bq3F//PHHQb9x48Z52drTOSSa/VkmTJiAxkZa6Lj9zrHMxbGsn2n96sImzefEPiUx/4c8ZV2OEbvGWVMLZM0YW5v3Zw17F+G9yqaaYJ8cvmdyhnUgvP+tXLnSy9bHkv197P2e4XswZ8hv165d0E+pCUJmzZrl5c6dOwdtfO35d8zC98LYHuN+/Du5ZMmSoN/48eO9zL+ZDYlWihBCCCFygR56hBBCCJELam3eeuWVV4Ljyy+/3MtcTI7VnUB69lVb6JHNZ1adyuo0VsHZUGlWpz300ENeHjBgQNCPwydZjRvLLsnZlFevXh20sWrRmtxYtciFSRtDJsvawqpsO89p4coxs0ltsO9n0yK32YzRYl3qoshoVrNmmrnMzhOfk+Yw3fTz7rvvBv3eeOMNL3fr1i1o4wzN7Cqwww47BP34PjZv3jwv2yKlfJ+NwZn0uSjzBRdcEPSTSSvk+eef97I1LfN6iJkFs5qn0wqT2rVx6623elnmLSGEEEKIekQPPUIIIYTIBTU2b1Wrkc8///zgdTZhxApupmUr5mzHQGiqsmYrhovaLVy4MGi75JJLio7BKjcgzAjK5q0DDjgg6MfRDW+//baXbTE+Np1YVTurBfk62ciExkDWaKZYpB9nDuW1EjNvxVSwaW02QymbSGNmE0bRW1XEMi2nma1iEVWx61qbqD2+J3Cx2zyRZvoZOXJkcPz1r3/dyzZbOl87vrd26tQp6Pfmm296mdeDjSBil4D27dt72d4/2SzG2Zn5ngsAPXr0gFgLRwDbqgh8X8salRWD9yKvGxvxzNFb5YI0PUIIIYTIBXroEUIIIUQu0EOPEEIIIXJBjXx6KisrcffddwNY13+Gwx05hNFmK7b222qsLwXb5a1tmG3Ka9as8TLbiQHgjDPO8PK//vUvL9sK5vPnzy967pMnTw76vfDCC15Oy0gJhP5J1peEYbur7VcdWhp7f2MhLYM2EPoAxEIp0/xu2H/K9uM5sn4j1uZdjU2xINaFM5jb+UzzF7Cvb6h/lJ0/Hs/6poi1sF8NAPTq1cvLdi753mN9Lpk0P7jYHmbfSRtGz75EaX5FgHx6LJz2xKYLyBqKHrtnpsHrhn+PgTBDM68h+5tZn0jTI4QQQohcoIceIYQQQuSCGpm3NtlkEx9abU1ObMZi1VXXrl1T+7Ga3GbrbNWqlZe58J0dg9WktpAom06OOeYYL++2225BP1YLsvnNquA4mzCbVWzYLhd3s+aptLBsq/6vLrIaUys3FrIWp62NCjbNTGXHiJlXeC6tejbtPXkmFv5aG/V4VmJznZZhW4Tme07PAYSmQM6EDITzzHs4tkdi6UrS7mW2MCmbRNiVgTP9izBjNhBeH5sCha99WlUEINyzWVOI8NiHHHJI0O+f//ynl9ldpCGzM0vTI4QQQohcoIceIYQQQuSCGpu3qs1aVnXZpUsXL3MElFVJsomobdu2RWUgVK1atSi3sXrWFv5kVXvr1q29zEX2gFCty+Y46wHPn8Xna9XurGq3bawaZjVuy5Ytg35Tp04FEBYobaxkzfKZ1RyS1XwRy+bLbay6bwrXu9TEIgrT1OOxbMq1wa4V3nN8/xFhdJS9b/O91M4r3+/4PsZuCRY2udh7X1pR2O233z7ox5mX+T0c0QsAy5cv9zK7Q+SF1157LbUt9rsT25c857weYpnXee+99dZbQT+ev1mzZnlZ5i0hhBBCiBKjhx4hhBBC5AI99AghhBAiF9TIp2fzzTdH7969AYQh4ADw97//3csdO3b0MlcmB8KwcvbBsfZktkFaGzLbg3k8mxmU7Y4cFmnDNtnGybZLOx77I6WF6Nt+LANhODvbQjmsFFibXdpmHC4nahOSXFvfjjQ/npi/UCxkPa3afVb/ozzDezWW6bquQ8d5zqyPAe+TuXPnerlPnz51eg6NEb6P2f3H90Xrz8b3Xb5v2WvP90++L1q/Er5PcvX0/v37B/1efPFFL/O92t6P2X8ojz49Tz75ZHDcpk0bL9vfDZ4zni/rB8t7lq+37ceZsnme2U/Vfu706dOLfIv6R5oeIYQQQuQCPfQIIYQQIhfUyLzFXHrppcFxtdkLAP7whz942ZptONSbTT82KyerYW3IelroYyzrbiw0k01psfEYbrPnzipeDqsEQtUiqwK58B8AnHrqqQCAG264IfUcGpqsGZRZNR7L5srY0No004ZV19v3pZ0fnzuPl9VclmcWL16c2sbzkRa+DmTP3JxWhNbuTVaxs5pfhFnm7b2P78czZswI2nivckoNOwZf+5jLArsicOHTI444IujHvws8hs1AnFboNC+wGRcIf3esmSktfYvt98QTT3h52LBhXm7evHnQj02hNpN3Wr+ZM2em9qtPpOkRQgghRC7QQ48QQgghcoEeeoQQQgiRC2rs01NtY7c2+sMPP7yoPHr06KAf+wJxdXObYpxt9tbPgkMpYyGyXGmW/QZshXi2NbN9Mmv4MvusAKGPj/U5Ofjgg728yy67eLkh03LXJ/Z6sD8Nz5/tx8dpfh52DMb6jaSFzitkff3wfrHpJPg687W085LVj4pDb7mfnXf2JeFSMiIsBWTXPft3rFy5Mmjj681pSKyvDpfradGiRepnpWF9Qng8Xk88NgC8//77Xt5pp50yfVZTgn1uAGDMmDFetvuN90us1E6af06s1FKsH98rdtttt9TPrU+k6RFCCCFELtBDjxBCCCFyQY3NW2khwWkccMABwfGECROK9nvzzTeDY1bJ2mrnixYt8vJ2223nZWtmstmgRd2SNYSbVeNcQRkI1aG8tuw6Y5U6t9lz4OOslaEZhayvn4EDB3p59uzZQRubSFi1bWH1O89T1mvMpg0gXBN5NHXE4KrzNr2GDQNnuOI231ttqDjfqzkE3la7534s29DrtNQEdm1wiHYeOfvss4Pjc845x8vWvMVmTJtRm0n7fbdpIHif89pYtWpV0I+Pzz///NTPrU+k6RFCCCFELtBDjxBCCCFyQa0zMtc1O++8c/SY2XXXXUt9OqIOYVWoLVzHZifOHGvNTBwJktVUFSskyhF8nHnWqtrTzgGouam3qcAmktNPPz1oe+GFF7xcWVnpZWvqYBNJrKguzxvPZ0VFRdCPzejWhJN32KS8/fbbB21swrLweueIH2u25MjT+++/38vWDHbggQcWHdvuK75f8Fx269Yt6Lf//vunnnse4SzXNsM/YwtkM8uWLSv6us3czOuG96g1OY4cOdLL7IrSkOTzri2EEEKI3KGHHiGEEELkAj30CCGEECIXlI1Pj2h8ZK2y3rdvXy/37NkzaOOKyjFfHbb7c9bQWPX0tHB4IPQjYR8CDse25NWHx8LX2Pp3DB06tOh7li9fHhyzjwBnY7fzue222xaVs4bDK80AcMstt3jZZszlfXXiiScGbezfxv4Y7777btCP/YT69++f6ZyOPfbY1Lbjjz8+0xgihDMe25D1cePGeXnWrFlethUT9t5776Jjn3feecEx+/7wuuFqDOWK7uJCCCGEyAV66BFCCCFELnBpBRqLdnbuAwALS3c6ogjbJUnSdv3daobmssHQfDYdNJdNizqfT81lg5E6lzV66BFCCCGEaKzIvCWEEEKIXKCHHiGEEELkgrJ46HHOHe2cS5xz6bUnwv4LnHNtiry+ulj/yDg16h8Z50znXMf192z6OOdaO+emFv4tcc69R8ebRt5X4ZybkdJ2pXPuoJS2da69c+4k59zPnXNDnHN7FXufWD+ay3zjnPuyMNcznXOvO+d+4pwri9+MvKO9WXvKJU/PyQBeKvz/ywY+l9pwJoAZABY38Hk0OEmSfAigNwA4534FYHWSJH/YwDEvL/a6c64Zil/7oQBuBHAkgNUAxm/I5+cVzWXuWZMkSfX8twNwP4CtYO7RzrmNkyT5osj7RYnQ3qw9Df7U7pzbAsBgAP8H4CR6fYhzboxz7hHn3JvOufucyTTmnGvunHvaOXd2kXEvdM5NdM5Nc85dEfn86wt/yTzvnGtbeK23c25C4b0jnHPbpL3unDsOQH8A9xWespvXyYVpwjjnejrnXi1cr2nOuR6FpmbOudsL8zGq+lo65+4qXOdqLd81zrkpqHpIDq59YY30BrAcwLkAflRo26fwV87owmc+75zrSuPf5pyb5Jyb7ZwbVt/XpLGiucwHSZIsA3AOgPNcFWc65x53zo0G8LxzroVz7s7CWnjNOXcUUHx9FPr+21Vpj2Y4506MfrioFdqbxWnwhx4ARwF4JkmS2QA+dM71o7Y+AC4A8HUA3QBwusgtADwB4IEkSW7nAZ1zhwDoAWAgqiamn3Nu3yKf3QLApCRJegIYi7V/wdwD4OIkSXoBmB57PUmSRwBMAvCtJEl6J0myBmJ9nAvgT4W/IvsDWFR4vQeAmwvzsRJAWtrWD5Mk6Zskyb1Y99r3AfB6kiTzAdwG4PpC2zgAfwZwd2H+7kPVXynVVKBqvRwB4DbnXHrKX8FoLnNCkiTzADQD0K7wUl8AxyVJsh+AnwMYnSTJQAD7A7jWOdcCxdfHYQAWJ0mye5IkuwJ4pp6/Sl7Q3ixCOTz0nAzgwYL8YOG4mleTJFmUJMlXAKai6oJV8xiAvydJck+RMQ8p/HsNwBQAO6Nqoi1fAXioIN8LYLBzriWArZMkGVt4/W4A+6a9nvlbCuYVAJc65y5GVT6F6gfF+UmSTC3IkxHON/NQyutA1Q316ZS2PVGlogeAf6BKw1jNP5Mk+SpJkrcBzEPVmhHrR3OZX55NkqS6vsghAC5xzk0FMAbAZgC6ovj6mA7g4IImYZ8kST4qMrbYcLQ3i9CgDz3OuVYADgBwh3NuAYALAZxQUJ0BwGfU/UuEPkgvAziM+gZDA7i68OTZO0mSHZIk+VuGU1LSohLgnDvGrXWy658kyf0AvgFgDYCnnHMHFLrG5pv5JPJxhwAYVYvTtHOvtVAEzWV+cc51Q9VcVhde4rlzAI6le27XJElmFVsfBa1+X1Q9/FzlnCvqSyJqhvZmNhpa03McgH8kSbJdkiQVSZJ0ATAfwD4Z3ns5gBUAbi7SNhLAWa7KXwjOuU6uyhHPslHhHADgFAAvFf7qWOGcqz6H0wCMTXu9IH8MYMsM55xLkiQZQTfDSYWb57wkSW5Elcau1wYM7699QRu3ccHJL2grMB5r/ca+BWActR3vnNvIOdcdVabUtzbgnJosmst84qr8HW8DcFNSPKPtSAA/qP4j1DnXp/D/OuvDVUUBfVowm1yLqgcgsYFob2ajoR96TgYwwrz2KEITV4zzATR3zv2eX0ySZBSq1GuvOOemA3gExR9KPgEw0FWF8B0A4MrC62egyiY9DVU+Qet7/S5U2SflyJyNEwDMKKjCd0WVr1RtuQuFa4+qv2qeo7YnAFT/9bMPgB8A+HZh/k5D1fqp5h0Ar6JKZXtukiT/3YBzyhOay6ZL88L1nomquRgFIC0o5NcANgEwrdD/14XXi62P3QC8WnjtlwCuKuF3yDPam0VQGQrRZHDO3QHgjiRJJtTwfXcBeLLglC7KALlFSkYAACAASURBVM2lEOVJY9+b5ZKnR4gNJkmS7zT0OYi6QXMpRHnS2PemND1CCCGEyAUN7dMjhBBCCFEv6KFHCCGEELlADz1CCCGEyAV66BFCCCFELqhR9FabNm2SioqKEp1KOl98ERbwXbVqlZcrKyu93KxZs6DfZputLeux0UZrn+/seJ98sjbxZIsWLbzcqVOnoB+PUV8sWLAAlZWVxbJObxANNZd5Z/LkyZVJkrSt63HLcT4//vhjL3/ta18L2jbddNNMY3z22drksZ9++qmXt9lmmw08uw1He7NpUYq9qblsGGJzWaOHnoqKCkyaNKlGH26jw4pXjYizbNmy4Hj06NFevv32tbVGt95666DfLrvs4mW+6a5YsSLo98orr3h5jz328PJvf/vboF/z5tnyDvJ3rs33Zfr3779B70+jNnMpNhzn3MJSjFsX85kWyVnbNTx27Fgvd+/ePWjr3LlzpjHmz5/vZf5+xx9/fK3OqS7R3mxalGJvai4bhthcliRPT9YffdbS/OlPfwranntubcLH//43TNrI2pjPP//cyxMnTgz6DR8+vOjnbrLJJsExa3T+85//eHmvvfYK+rVq1crL++23n5d/8IMfBP3K4a9QIWoK79uYVnPRokVevvPOO4O26667zsuska0L+JxOO+20oO2aa67x8vnnn48sfPXVV6njCyGaJtrlQgghhMgFeugRQgghRC7QQ48QQgghckG9196aO3eul4cNG+blbbfdNujHTsnWB4ejtNhB2ToWrl69er3vAUK/oA8++MDLNsqLI0meffZZL7/88stBv+9+97te/uY3vwkhypGsPi19+vQJjt9++20v854AgM0339zLvKetXx77vfFef//994N+a9as8TIHEtjxfvrTn3qZAxAOPPDAoN/999/vZft9+XrIvycd6/Cedt1i/pyx8ke1cZwfP358cMz+mG+99ZaXd9xxxw3+rKZMXQczZOXUU0/18o9//OOgrW/fvl7m+439Hc+KdrYQQgghcoEeeoQQQgiRC0pi3oqpwn72s595uUOHDl62Yd5sWrLjbbzx2tNmdRybs4BQ/cUym7OAMDkhm9L4c4Aw2SGrdO14N998s5cPOeSQoG2LLbaAEA1F1rD0Pffc08szZswI2tq3b+9lu/Z5r3Kb3UtLlizxMpu0bC4sTmLIJi3ei/aY7x0PPPBA0I8THP7rX/8K2vh61GWurTyR9VrV5pqOGTMmOJ4+fbqX2eQKAJdeeqmXeS5HjRoV9KutiaQcybpmY/34mPtlzbf3v//9Lzjm31Oer+OOOy7oN3v2bC/b33Hep3WxF6XpEUIIIUQu0EOPEEIIIXJByaO3bDQGq7W32morL1u1GKvDWSUNhOaoL7/80su29hYfs+raRn7w+NwvFjXGZiqraufze/zxx4O2U045BUI0FDH18IgRI7w8YcIEL3fp0iXox6Zdu295/DQZCPc+q85tRFmaOc7uYR6f923Xrl2DfiNHjvTy008/HbQNHTo09XzzQFYThn3d3nfTuOeee7zM5X7GjRsX9Lvxxhu93LFjRy+//vrrQT+OxOIIHwC44YYbvNy7d+9M59fYSTNNxfrx76eF96KNZGYzNPezv5kvvviil4855hgv29p7O++8s5fZPcRix68N0vQIIYQQIhfooUcIIYQQuUAPPUIIIYTIBSX36VmxYkVwzD49bAu2mV3Zz8bajDkUNi3MFAhtjWzHtPZJJmYXZT8jztzcpk2b1PPjavGAfHpE/RPze2M4eziv6Y8//jjoF8uWzj4+sT3HbVmzH8f6pd0HbEg9n/vhhx8etLH/IWeTtuduw+/FWmbNmuVle9045HzSpEleXr58edDvjDPO8PJ+++3nZeu3w2OwDIQ+I3PmzPHyDjvsED3/pkJWn7TY/YDbYr40vPfefffdoI332JZbbull60t03XXXeblTp05BW12nj5CmRwghhBC5QA89QgghhMgFJdfTTps2LThmlSebumyoKh/bkHAOY+zevbuXKyoqgn5c/JBD7Fq0aBH0Y9Udm9k4gyQAPPHEE0XHW7lyZdCPM0py+LoQDUGaCvuoo44Kjtn0wykZFixYkNrPmpzS1OCx0NjaYD+X1d78fe19he8J9r7C5peTTjqp6HhNmaymA5tChIt9slmwZcuWQb+zzjrLy9dff72XrTmDC04uW7Ys9fw4zHnKlClBGxeE5nnOi3krazFhy9KlS73MZscPP/ww6Dd58uSi77EmzVatWnmZ18ZHH30U9LPFwkuJND1CCCGEyAV66BFCCCFELii5eYvVxACwzz77ePm+++7zsi1qyAXjWI0Zw6pd16xZU1S2JifO7sqmLxtpdfXVV3t5wIABXmYzHRCq0OfNm5fp3IWob1555ZXUNhtNycRU5bEszEwsY2wWshZKtOfK0WU2q/PEiRO9zPetvGRntiZIvnZ8DWKFnfk+bguE/uUvf/HyM8884+VDDz009ZzatWuX2samLzajAMB7773n5TvvvNPLe++9d9Bv1113TR2/MROby7lz53r5ggsuCPqxqwZHW82cOTPoxy4mb7zxhpeHDBkS9GPTJd9TbKHXWER1VrKa0KXpEUIIIUQu0EOPEEIIIXKBHnqEEEIIkQtK7tNz0UUXBcdsW9x///293KdPn6DfqlWrvGx9ethmz9WaW7duHfRLyxxrbfQ8HofSWT8jDndkfyQO77XnYW2Xeae21X/T/Atqmy2XQzqzhnNa2D+EP7ex+IBw2gUgzF4cu448h7GMzDxGzN4eCzFPWy+xMHJeEzYsnf0KbOqK+++/38ucITYvxNIAMHbd8ByNHj3ay6eeemrQ77bbbtvQUwzgMGr+vQCAfv36eZmzM1tfNRuK3VSIZVDmNC933XVX0GZ/Q2tK27Ztg2P2m2P/qRNPPDHoxz5CsXs/t8UqJsSQpkcIIYQQuUAPPUIIIYTIBSU3b9lwxOeff97Ljz76qJdHjRoV9OOic7fcckvQxiYoLiZnQynTzCCsggdC9Ser0qx6lkP4fve733nZmrC22WYbLw8fPjxo4+ylNswyD2Q1/VjVZdr7sqo07Rq66qqrvLx48eJMY1hiKuRy5fXXX/cyF80Fwgy6rJbm/WHbrPkorbipNVtxWyzMPa3YYKy4MK8J248LINt9m/dColn3Jt8HAWDfffctKls4bQivm6ypDWw/LhDL91wgdHsYOnRo0fcAwMKFC1M/Ow9YcxbvI97LWe917LIChL/xPEdjx44N+l188cVezloE1ZLVVClNjxBCCCFygR56hBBCCJEL9NAjhBBCiFxQciP2JZdcEn4g2c05TG2XXXYJ+j3++ONevvLKK1PHZ1ujtdGn+Q1Y232av48tV8Eh8IMGDfIyV48FQrumreqbRz+eGGk2+6z+FRxmDABTp0718sMPP+xl63vCoZUnn3yylx944IFMnwuEId6///3vvfyLX/wi8xj1Da9162fDsH+cDWXmObMpA7iNx7e+NewvwOPHQtZj9vy0fjb8le8X9nstWrQodXyRTta5ZLittlXs2SfNpg1JW4fW7zPvflwx38mYHw/ve76Gp59+etCP78H8WeyLC4T+XjYlAsMlL77//e8HbVzyIoY0PUIIIYTIBXroEUIIIUQuKLlu75hjjgmOOWR98uTJXuawQgD4xje+4WWupgsAXbt29TKrVm0oOqvMYhlhWT3HFdKteu/jjz/2Moc6Xn/99UE/brOVhjnztM1C3VSJhZ2mhau+/fbbwTGrSbk6uE110K1bNy937tzZyzbMdsGCBV5+6qmn0k49yoMPPujl//znP7Uao76ZMmWKl9k8B6SHhNuQdVY/WxNwmkrcznNahm1rcuJ9G8vEnba/7et8T7DZY9lEwvPJpmyxLmnmKfs6r5vY/Th2v2B47d19991B27Bhw7x8yimneNmawWKmlDxQ2+zxaVns+boDYZg6V3DnlAJA+FzQpUuXoM0+Q1TD6SeA0NWBKyZYpOkRQgghRC7QQ48QQgghckHJzVuzZs0Kjtl8xFFPe+yxR9Dv5Zdf9vL06dODNlbJxSIE0jK9xopepkUi2PNllWnv3r2Dfttvv72Xrapup512Sv3sciRWmJPNI9YEwsRUqKzyvPTSS7380EMPBf24OGSHDh28PHDgwKAfmzg//fRTL9uite+9956XL7vsstTzY9OqPacf//jHXn7zzTe9zGZbICx+2NDw2rf7gM0RWTOw2jH4fZy52Zo60sxWsb3J2DXFhSQ5s7SN1mGzmP2OPMYNN9zg5ZpE9JU7WTOdl5pYhF1aPwtnE7auApMmTfLyd7/7XS/PnTs36LfXXnut/2SbGFnNh7F7RdZ1w79/7B6yfPnyoN+RRx6ZOkb79u29zHvWZn/m34UY0vQIIYQQIhfooUcIIYQQuUAPPUIIIYTIBSX36bE2VLbfvvvuu162WY1joeMcdsi2RptdM80/J1bJmf1A7Oeyfwefn/UbYH8R9lkBgCVLlniZw6vLiZgtl4n58TAcjshVd4EwzJCzVffs2TPox3P70UcfeXnVqlVBPw5BZT8gtvED4Xrj8MZrr702dbzddtstaGMfEPZfseHx5YQN2WXSqirbeeY1EfPHYGK+d1mJhdHzPuP9bcPyOau6PScek+ezKdFQPjwxsmZk5mzrALD77rt7mbOqA8CTTz7p5ZEjR3rZrgfrc5kHarMG0kLU18frr7/u5V69ennZVrvn9B/2nn755Zd7mX9rDz744FqdkzQ9QgghhMgFeugRQgghRC4ouXnLmke48CObLKxJgM1MVrXGamlWr9vPSgu3tv3SiuRZVSi3tWnTBmlwOJ7NHLt48WIvl6t5i9WfWVXPN954o5dvvfXWoG3p0qVeturkXXfd1cu8Hvg9sfOLmSp5Xm32XatCrcaGsI4YMSL1PK666iov33zzzV7ebrvtgn733ntv6hj1zW9/+1svW/MtH7PpzoaXcqhw1hDzuoD3ujVv8Trlc7dZ2tm8x/cYIDRZ/+tf//JyuYR5NyV4LmP3mGuuucbLdh2ee+65Xv7HP/4RtPEaPfzww73MmdiB7Cb6vJAWzm5/x9KKedu9wkXA+Te+JveN3/zmN17m3+Djjz8+8xiMND1CCCGEyAV66BFCCCFELii5ectGSKSZH7gwGRAWBoyZt2Kq5qwZmdPU+lalx5/LWSLZZAeEqj87BmelLBe4CCUAPPvss15+6623vGwjWthUx9+LI2SAsPAnR14B4fW2bQybHviaxkyVbNqwa4ijsnj+bOFQzvJpi2t26tTJyzvuuKOXrdnk9ttvR7kwb948L7PqGQjngk271lzH368+zVtMbA/zWrTmrVg2dza5VFRUFH2PqBv4HmlNTr/61a+8zHu9Xbt2QT+OBO3Ro0fQxvPO96nGaM7itc5rNrb37P2uttFXae9P2xP9+/cPjjlrMkfRxbBuJbwv+V4UczGJIU2PEEIIIXKBHnqEEEIIkQv00COEEEKIXFBynx4L22jZLmgzMlu/iDTSfITsZ7Et1Nry+Thr9V/2h4iFyseyRDcky5Ytw0033QQAGD58eNDG/lSxLLhsN+fsx/Z6cBZNO0fsq8O+QNYXitcK+xbZz2K/FJ4H/k52DLYhc4VuIFwP1u+M/Uh4/HLz2+IM4Xye1iaelo3czllapnMgPeTVhiVbu30aPD6PEQuNZd8wu2bZf8vOE+/Vd955J9P5lQv2vpI11URdfzbPi51j3uuzZs3y8oUXXhj0Y/84ztp/3XXXBf1ivlacvZn92Pbcc8/U95SaWOqDWOXz2qQQqWtiPkHf/OY3vcxZlwHg73//e9H32N9gHt/e+9mXsk+fPus/2fUgTY8QQgghcoEeeoQQQgiRC0pu3soa7mlNB1bFxaRlV7ampLTQ9tg58RhWZcyfxWYCG6LNJhZLuRQybN26NU477TQAwIABA4K2l19+2cszZszw8sKFC4N+bB5YsWKFl22YMF9Tq9bkIq6VlZVejplUWG1uPystjNMW2mRzHJtArPqY14pNTcDnwap7Gwp+xBFHePn3v/990fMrJePGjSv6eszkxOYt+705M641H6Wp4rOmlqgtfM15bu06YlOrvcfw96yLAqn1SczsEQttrotrn+YSwHsCCM2sf/zjH718wAEHBP04bcTDDz9cq3Pi7xU7p/oklj2+NvPw5ptvBsd33nmnl63J0GakryZmZuLfKnsP+MUvfuHlDz74wMvWVSKNmLkslqKme/fuqe/Lmj5Dmh4hhBBC5AI99AghhBAiF9R79FZWWLVmVbdpGSpjKumY+jCt4Kg1U6xcudLLbN6y2UA5csCq/xsqg20xqs+Fi34CwKBBg4r2t2a7+fPne3nOnDlethlWOSOqNe+lzaVVcXIBQS5cx68DoamRI7GsCZLV3DGVN5t8YnPHkVBsXgEaPqOvLSxajV3fadleed0DobkgZlJO21f2mM8vdo35c+01TTPH2e/OZlhrvrbfpalQ1+svFoUUM7NxpuWOHTt6edq0aUG/hx56aAPPMFx7bDav74zMSZJ4E3wsezyvPTYdAcAdd9zhZRvlzPD9+LHHHgvaOLN+2jnYc+R9xFF0QGh2fOqpp1LPiX8nOQt+zKzGexQI19fgwYNTP0vmLSGEEEIIQg89QgghhMgFeugRQgghRC4ouRGb/S+AMGQ05oPDtkBrl2e7cSz0LS3jpbX9pYXHx/xx+Ny7du0a9Js0aZKXrd9EuWRkbtasmfdzsdXD33//fS/H7KStWrXy8pAhQ7xs/XbSfEqAdD8NuzZ4zLTwdSAMYef38LoDwjDLWFVuPne7TjiDMa9z6xtiq5TXN/vtt1/R162vR5qPgZ0LviYxvyAe3147PmZbv73+aeHQdjw+p1jGaB6/obLbloKYnw37ZC1dujTox3ud93CMrD5Cv/zlL4NjXlPsxzNixIhM48XSmMQy37NPT33jnIve/4oxZcqU4JjnLHaP5Cr0nAoEAJ544gkvH3nkkdHzLcbJJ58cHB922GFejoWR897OypIlS4Jj9pHca6+9ajyeRZoeIYQQQuQCPfQIIYQQIheUxLzFJodYFsqtttoqdQxWQ8dCSXn8mGo8ayhszHSWpq6vqKgI+vF5xNTr5YINsbbHabAJMmY2YNOSDXtPux7WDJhWFDb2Pp4va2bt1KmTl3ltWBV67HulrRt7/Tg8tyH497//XfR1a77lYzb/tW/fPrWf3Vdpa99eOzaLpZnEgPAax/rxvMUyK6fNWbHjxkTM5PTGG2942YYe8z3YFnmuTfZizro8fvz4oI3NzWlZwmPEzLGxvg1ZPHb16tV48cUXi57Hcccd52Ves2xytHAaDlvFgE1J9h50/vnnezlm3mKOOuooL8+cOTNosyHxdQkXDAayr0OFrAshhBBCEHroEUIIIUQuKIl5K1bck9XfbGKwxLKvpqk1rXorLWLLvj8tc6z9XDazccSPzcgcM2+VU0bmDYXVqTEvfauGFfXLM888U/R1azZmkxOv71tvvTXo961vfcvL1jzJhV157VtTGrfF9nrae2yEIB+zetxGrnHRXJulOw0b8WTNfaWg+j6RNVIqFr1VFxEvWTn77LO9PHv27KDtySef3KCxY5n5LbxWbGHO+uSzzz7DvHnzAADf/e53g7bLLrvMy7xv2ERo2zgSzJoq+X2xop0XXXSRl7/zne8E/S6++GIvv/DCC14+6KCDgn42E35dYs171jUhjax7RZoeIYQQQuQCPfQIIYQQIhfooUcIIYQQuaDkGZmtnY1ti7FQ3qxZVdNCWou9r5qsVYJjNmP2G+jZs2fQFqv83pR8ekTjgNMEsH3chiin7ZdjjjkmOP7hD3/o5fvvvz9oY1+g5cuXe7lDhw6p58RYvw3em+zPYDNs8/sGDRrkZQ7VBYCxY8cWHbvYZ1fz+OOPB8fst1IqaloZPdaf7zmHH3540MZ+IJdccknQdsopp2T67CuvvNLL7D92wQUXBP122223TOPVBfy7YKt21yetW7fGmWeeCQD461//GrRxKgE+R7sPubI6r3vOtA0Abdq08bL1eeM1cO211xaVAaBt27ZeZj/NK664Amnwb1wsjUBW7PfK6nuX9bOl6RFCCCFELtBDjxBCCCFyQb2bt1jNFivEyOGzrHIDQhV9LItqWtHEWKFTPj+rgk8rYBkLvbfnFyuaJ0Qp4D3I5qesamPL7373u6JyDKtu5/PgPWfvF3zMYe+xbO5ZiWWT5gy5XKwRKL156+OPP8aYMWMArBvqz/c+LvhrM/Dy/ZO/C8sAMGfOHC9fd911QRuHKXMxy1GjRgX9/vSnP3mZi5ZmXRu1JWbS43u8LYrbUNjM/RMmTPAyF622RZQ5ZQJ/Lw5lB8Lfq9i14RQisWvDZrWYabKmplhg3d9WNqXZjMxpKSLsPcWu7TSk6RFCCCFELtBDjxBCCCFygR56hBBCCJELSuLTk1b+wRJLL802P2u749DVDz/80Ms2rX7W8HOGbabWb+CTTz7xMqfKtrZEPnfrw2PttUKUmr/97W9eHj58uJd5PQN1H3rK2D2S1f5e17BfBVeSB0IfJ77n7L333iU/L+bzzz/HggULAMD/X82yZcu8zH5RfE8EQr8Nvg926dIl6Hfqqad6uVevXkHbc88952WumD59+vSg3+DBg73MfkHWH4nvi6X2s2EfkUMPPbSkn5WVn/3sZ8HxAw884GUuKWF/q/h3kn+T7DVk3xr7u8P+ajy+9W/lNWXTUTAbeq+I/R7b3/s0n56Yb24MaXqEEEIIkQv00COEEEKIXFAS8xZnw7Qqzqwmp+OOO87Lq1atCto4hJ0/Kxa+zv1i1dhZVWfNZS1btvRy//79Uz+LVc32nPg8hKgP2GzDVcZt9W3eZ1mz8caIpYng41jIa1qbVanzcSwE/rDDDvPyHXfcEbRxGoojjjjCy1x5uj7gLL5ZYTM/ACxatMjLnBmbXwfCa8VrAwhNWrw2bFZnXivWfMbUZ+g4m7f++Mc/epkrm9c3Nuybrz1nsr788suDfhMnTvSy/S2sa/bZZx8v77///iX7nJhJjNcdkF65oTah8oA0PUIIIYTICXroEUIIIUQuKIl5a82aNV6OqbVtYTHGero3JljtZr9/7DsLUWpimV85csOaQRiO+rKZgBlWYdd1NFgMNiFbE3Xv3r1T29i8dd5555Xo7EpD69ato8d5g6P0GsNcstmVZcvs2bO9PHny5KBt2rRpXuZCskBo4uTfJ1tN4Lbbbiv6udYlZEP3c8zUedFFFwXHO+20U9F+1nUmK9L0CCGEECIX6KFHCCGEELlADz1CCCGEyAUl8enh6r877rhj0MYhjYMGDUodIxbOXttQtfqCQzjnz58ftPXr16++T0cID++ra6+9NmjjfduhQ4fUMcqlanUasfsDp7vgsGYg/F716YMkSsuvf/3rhj6FOoN/T+1v68knn1yyz63r39zYeAcddFCmMWIpamJoZwshhBAiF+ihRwghhBC5wGUtxAkAzrkPACxcb0dRl2yXJEnb9XerGZrLBkPz2XTQXDYt6nw+NZcNRupc1uihRwghhBCisSLzlhBCCCFygR56hBBCCJELyvahxzn3pXNuqnNuhnPuYefc5uvpP8Y5178gL3DOtamfMxVZcM793Dk30zk3rTCv6fkKaj72EOfck3U1noijvdl0KcU+5fnfkD6i5mg+16UkeXrqiDVJkvQGAOfcfQDOBfDHhj0lwFUlGHBJkny13s4CAOCc2xPAMAB9kyT5rPCjV7vCKXWMc27jJEm+aOjzaGRobzZBynmfipqj+SxO2Wp6DOMA7GD/onfO3eScOzP2Rufcjwt/kc5wzl1QeO13zrnvU59fOed+WpAvdM5NLDwZX1F4rcI595Zz7h4AMwB0KfZZIpUOACqTJPkMAJIkqUySZHHhr/4rnHNTnHPTnXM7A4BzroVz7k7n3KvOudecc0cVXq9wzo0r9J/inNvLfpBzbkDhPd2dc/2cc2Odc5OdcyOdcx0KfcY4525wzk0CcH79XYYmifZm0yFtn15euO4znHN/LTxcVu+jawr7dLZzbp/C682dcw8652Y550YA8FkgnXO3OucmFbQPVzTEl8wRms8ilP1Dj3NuYwBDAUyvxXv7Afg2gEEA9gBwtnOuD4CHAJxAXU8A8JBz7hAAPQAMBNAbQD/n3L6FPj0A3JIkSc8kSRSCWDNGAehS2Ei3OOf2o7bKJEn6ArgVwE8Lr/0cwOgkSQYC2B/Atc65FgCWATi40P9EADfyhxQegm4DcBSAdwD8GcBxSZL0A3AngN9Q902TJOmfJMl1df1l84L2ZpMjbZ/elCTJgCRJdkXVD94wes/GhX16AYBfFl77fwA+TZJkl8JrnIb+50mS9AfQC8B+zrlepfxCOUfzWYRyfuhp7pybCmASqn7A/laLMQYDGJEkySdJkqwGMBzAPkmSvAagnXOuo3NudwArkiR5F8AhhX+vAZgCYGdU3VABYGGSJBM27Cvlk8K17wfgHAAfoOpH7MxC8/DC/5MBVBTkQwBcUpj/MQA2A9AVwCYAbnfOTQfwMICv08fsAuCvAI5MkuQdADsB2BXAs4VxfgGgM/V/qO6+Ye7Q3myCRPbp/s65/xT23QEAetLbiu3ffQHcWxhzGoBp1P8E59wUVM1jT4R7WNQhms/iNAqfnmqcc18gfFDbbAPGfxjAcQC2xdofQAfg6iRJ/mI+twLAJxvwWbknSZIvUfUAM6aw2c4oNH1W+P9LrF2PDsCxSZK8xWM4534FYCmA3VG1Dv5Lze+jaj30AbC4MMbMJEn2TDklzWft0d5sohTZp99F1V/x/ZMkebewB3lui+3fojjntkeVNndAkiQrnHN3YcPWiVgPms91KWdNTzEWAvi6c+5rzrmtARy4nv7jABztnNu8YB45pvAaUHUzPQlVN9eHC6+NBHCWc24LwraqAgAAIABJREFUAHDOdXLOtavrL5E3nHM7Oed60Eu9Ec9SOhLAD8jW3KfweksA7xccVU8DwBXnVgI4AsDVzrkhAN4C0NZVOfPBObeJc47/ohF1i/ZmIydln1b/4VFZuPbHZRjqRQCnFMbcFVU/sgCwFaoeUD9yzrVHlWlUlAjNZ3HKWdOzDoUn03+iymFxPqpUarH+UwpPn68WXrqjoD5HkiQznXNbAngvSZL3C6+Ncs7tAuCVwu/tagCnouqpV9SeLQD8ufBj+AWAOahSuQ5L6f9rADcAmOac2whVcz0MwC0AHnXOnQ7gGZi/8JMkWeqcGwbgaQBnoWpD3+ica4mqtX4DgJl1/N0EtDebCGn7dCWq5nUJgIkZxrkVwN+dc7MAzEKVqQRJkrzunHsNwJsA3gXwcp1/A8FoPougMhRCCCGEyAWNzbwlhBBCCFEr9NAjhBBCiFyghx4hhBBC5AI99AghhBAiF+ihRwghhBC5QA89QgghhMgFNcrT06ZNm6SioqIkJ/LVV2Fh5Pfee8/Ln3wSJlxt3bq1l9u2bVuS8wGAFStWBMeVlZVe3mqrrbzcvn37kp3DggULUFlZ6ep63FLOZan573/XJmJetWpV0Nas2dp8hRtttPaZfosttgj6bbLJJiU6uziTJ0+uTJKkzhdtY57Pxor2ZtOiFHtTc9kwxOayRg89FRUVmDRpUt2clcE+2Fx22WVeHj9+fNB2+umne/l73/teSc4HAB5++OHg+I477vDy0KFrk09ecMEFJTuH/v37l2TcUs5lqXnrrbXVKZ555pmgrVWrVl7ebLO1GdH32issyN6pU6cNPg/OcVVImLdenHMlKYjZmOezsaK92bQoxd7UXDYMsbmUeUsIIYQQuaBBy1Cce+65Xh47dmzQxuYuaz5iLdCNN97o5S5dugT9evRYW3akZcuWXl6+fHnQjzVJn3/+uZet6aRDhw5evvXWW738xBNPBP1uv/12L3fr1g0iG1k1J//v//0/L7/66qtB2xdffOHlzz77DGl85zvf8fLrr7/u5U8//TTot++++3r5uuuuC9qaN2/u5S+/XFsNgU1sQgghygdpeoQQQgiRC/TQI4QQQohcoIceIYQQQuSCevfpGT16tJfnz5/v5T59+gT92J/GhrPvvvvuXv7ggw+8PHfu3KAfR4RxpMW0adOCfhtvvPYytGnTJvWcli1b5uXtt9/eyytXrgz6/eQnP/HyiBEjILKR1adnyZIlXt5mm22CNvbJ2nTTTb1s5+jee+/1MofA21D2mTNnepnXCRD6k/Hnsq+PEEKI8kGaHiGEEELkAj30CCGEECIX1Lt569lnn/UyZ6q04cVsZvjf//4XtLEJik0ObB4BwjBiNlNY8wNn691yyy29zFmhAWDzzTcv+lmdO3cO+rFp7qWXXgraBg8eDFEcNmNyNmUgNB+98847Xm7RokXQj0PW2bxpMzKzWYzNrGwSA8J5/tGPfpR67vZ8hRBClB+6UwshhBAiF+ihRwghhBC5oN7NW4sXL/YyF+2MmbfYTGX7sjnCmjDYJMLYjLlsjuKMvGzOsuOzOcOeH0ceybwVh81HNkqP4ag/NluxOTI2hl0LPAavJ2tK7dWrV9H3AGEU2bbbbpt6DjJ9CSFEeaC7sRBCCCFygR56hBBCCJEL9NAjhBBCiFxQcp8e69/A/jNc+ZxlIMySa2G/C/anWb16ddCPw5fZ98f6bfA58nvsufP7Nttss9TzY5+e2bNnp/YT4bWy4eLMxIkTvcz+M1tvvXXQ76233io6tvXP4kzeDPuZAcBRRx3l5VGjRgVt/fr1K3pONnWCEEKI8kCaHiGEEELkAj30CCGEECIXlNy8xdlugdBktGbNGi9bswJnzLXmqI8//tjLnJHZhiWzmYHNZdb8wOHxbN6y/dhcwmHI1nTC2KzOIiRrkdEXXnih6OvWvHXwwQd7ed68ealjs3mrd+/eXp46dWrQj9fUscceG7Rtt912Rc/JpkQQ2VmwYEFwvGjRIi8r3YMQYkORpkcIIYQQuUAPPUIIIYTIBSU3b73//vvB8de+9jUvs4nImpLYdGAzHnMWXn6fjd5isxV/Fr8OhOYzLkZqzRQcXdShQwcv20y9fB6tW7cO2tis0rZtW+Qdnls2VVrYVMVZsydMmBD0a9WqlZd5bdjowCFDhniZTSgnn3xy0O+3v/1t6jllNc2JOA8//LCXL7vssqDtsMMO8zKbMnfdddeSntO9997r5R133DFoGzhwYEk/WwhROqTpEUIIIUQu0EOPEEIIIXKBHnqEEEIIkQtK7tPz4YcfBsfsC/PRRx95+cUXXwz6fetb3/Jyx44dgzb2E+IK2eyPA6Rn+LW+I9yPQ9Ztv3bt2nmZfUlsFe1ddtnFy5yBGgDefPNNL8unJz28e9y4ccHxsmXLvMz+HHZ9rVixwsuc9sBmYOYMynPmzPEyz52oOZySgveFTd3wwx/+sGhbt27dgn7Tpk3z8jnnnOPl8ePHZzof6+d35513ermysjJo4xQaW2yxhZft/aepEkvREePGG2/0ct++fb3M90sgvGfyva9Xr15Bv06dOmX63KxcffXVXu7Zs2fQ9o1vfKNOP0uUP9L0CCGEECIX6KFHCCGEELmg5OYta1bgbMqcZdf2mzx5spf33XffoI1V3hzGas1ZrGrnMHWbuZlNWpy52Yaicxg9Z2H+z3/+E/TjMTp37hy0vf76617eZ599kHfSVOgcMgyEqneeL5sSgE2caZm2bT/m+OOPD45//OMfe/mPf/xj6rkrfL2KtGKry5cvD465MGxFRYWXYyYRvkfY9bH//vt7+cknn/TyiBEjgn5swrL774wzzvByqUPiyxGbGiQthcRzzz0XHJ900kleZrOVvfac7Zzvn7fcckvQj02cAwYM8DIX+AVCU7TN5P388897eeHChV7m+Qdk3sqK3de8Bni+unfvnvq+crkvStMjhBBCiFyghx4hhBBC5AI99AghhBAiF5Tcp+c73/lOcMxVsFeuXOllDnsEwtBSDvMGgM0228zL7MdjfXU4ZJZLTVj7JI/Btmb2PwKAV1991cucOt/6enAI7m233Ra0cRmOPGL9BtJC1keNGhUcs+8OX18uSQGE85yWsgBYN9S9mtNOOy31/I466qig7bHHHvNyudir6wr2h7PfLfZd0+Zzt912C465XMjMmTO9zGkGgNCPg+fsBz/4QdCPfed23313L//kJz8J+rGvDqfPsKT5kAHrlrFpTPC8AuE90vrwzJo1y8t8v+OyLQDw1FNPeZnnz16nrl27Fv0sWyKGj999910vT5w4MejH/kP23E844QQvc4qT2bNno6lSF/4zXO7nyiuv9DL73QHA2LFjvXzkkUd6mX0gN+Q80rjpppu83Lt376Bt8ODBmcaQpkcIIYQQuUAPPUIIIYTIBSU3b1k47Hv48OGp/VgNbbPzsio7LUTWwmpdq+Jlk8tWW23lZWsC4X6snr/qqqsynYOIqzs5FYENQd1+++29zFm42dQJAF26dPEyq2ptllebRbsaXp8A8PLLL3uZs4Q3BWKmjrTrU1dce+21Xj7wwAO9zCZDIMyMzOaR9u3bB/1Y7b3ffvtt8PnxOm0M5ix7H+RjltPMjwDwzDPPBMfXX3+9l8877zwv26zZaSajpUuXBsd8Tdks3aJFi6Afr0tOLWHXK68Nm2qC1y+byDhjO7Cuqa4cSfuNq4nZmc3+bE5+/PHHg35sCmSmT58eHHOoP19T+1tdm7QsnK4GAL73ve8VPY+jjz466CfzlhBCCCEEoYceIYQQQuSCkpu3rGouzcxkVcgc7cFqTCBU4/EYNsqCPfpj6np+H4/NkVxAqCaNYSOUmJh6OQ/E5oEjtux64Kg3VtXaOecCk2wGs0UjObsvf9Y777wT9LvssstSz/fMM8/08l133ZXar76o3msxNTfvx9hcLFmyxMv/+Mc/grann37ay6NHj67xeQLAoEGDvMyRNjw2EO7hNLMHEEYXxcxbvDe54DEQrh3O3Lt48eKgX3WEko0cbEjsfZbnlq8bZ8IGgJ122snLV1xxRdDGEbScnZ5NzQBw6qmn1vh8OXJ35MiRQRtnbmYTtTWDcfZfm9GfTWs8T/a+Uh/mreq5iRV0je3Z2kRA2fvYpZde6mVeD2wyBsIoLXbh2HLLLYN+bBbjqgg2CzdXK+AIXDsPHKFtz33vvff2Mrs9zJgxA7VBmh4hhBBC5AI99AghhBAiF+ihRwghhBC5oOQ+PdYeyT4tMZ8C68fDcKZdrmhus3Ky/T7ND8ieB49nbcixDL9p4zW1TL21gefB+jSx3w1n5bbZNtkXgTNv2zmxtudq2rRpExzPnTu36PlxygIg9NWx4exjxozxMlf2HjZsWNFzqC/s+s66Bi+44AIvc/Zxe004RJXDSYF1K2Zn4S9/+YuXH3jggaCNrzHb82229LvvvtvL7HvHGeCB0Idj1apVQRv7h/G9xPof9OjRA0DoA1RfpGXdtfdSnj+eLw7tB4ADDjjAy//+97+DNr7e7LfD/lOWtGtoYT+QE088MWjjY/bbuPnmm4N+zz77rJfZzw8I/bD4fmEzftcH1fOUdR/a/cvrrLKy0svW92X58uVefvvtt4M2TuXBGcvZfwoI74W8l+11O+igg4qeu70f837jfWmrJ7DPJmfaBkKfrMMPP9zLNiUC+53FkKZHCCGEELlADz1CCCGEyAX1npGZYVWaVYWyutK2sbqZVX82jJVNVfweqz7k8TlU1arqdtxxxyLfYl3qovBbUyIWps/ZrFn9yepvIFTPppm6gHVNklnOideDNRPwmmJTHBBmg+aii9Zscsopp2Q6pw2lpmp0S8+ePb183333ebnanFPNDjvs4GUbonrJJZd42YbDpsF7k1XvQKhi5+vPYawA0KdPHy9zugtbKHHgwIFFx7PwPcFmZm/Xrh2A7GutNlSvyaxZd2+99dbgmE1TPK9DhgwJ+rGJyLa99NJLXmazQuw+yOcXC9HOeo9kk7dNHcC/H9bcyXuQ7yXWbcKmsigl9ncnLUybzVRAmFqBTT3WlM+mRXvtv/71r3v5xRdf9DKHkQNhpvPqdQ6se0/jqgiMNTHxfuY0BXbv8O+4TQXBKRK4GC2bcIHQ9BdDmh4hhBBC5AI99AghhBAiFzSoeSvGe++952UbPcFmK8aq1tIKBVoTRpopLRblxV7pVtWXtQhqUyV23SwcHcVqaJv9miOI2HwxZ86coB9HqrBpw0baZC0iyeZOq07myJfaRC3VJUmSeFOfVQ+zSjhmSjj77LO9zFFU1uxx+eWXe3mPPfYI2ji7Lo9n53PChAle5qy7dm/36tXLywMGDPCyVY+zqYqj7CZNmhT04/NgdTsQmlB5DdusvdWmnlKarmta8NXeg9jcx2YPa6rkws72e/bt27doG0faWLJmnI9dO15Dt99+u5cPO+ywoB8XOrXRmZxNn9e/Pb9Sm7eWL1+Oe++9F0Bo+gWAs846y8scsWSjJdkExd/Tmuo4K7WNgGKTGUfG2vXA9zsuMmt/09Iy39tqBLbAazXLli0Ljtk0Ze/N/FlTpkzxsi1KnRVpeoQQQgiRC/TQI4QQQohcoIceIYQQQuSCBvXpidl1X3nlFS9bGx+HKbPt3dqa2T7Jbdauy/3YV8BW8OZ+bJO09nQ+p6ZcVT1rdljmiSeeCI7ZV4B9evhaA2HIJIen2hBnXhsLFy70srU182fx+cayyHbr1i04/tvf/pbat7757LPPfJZpW7Wa5ylWqZx9BNi3xoalcz+b1uGcc87xMvsR2Iy5/L6dd945+B4M+3FMnDjRy506dUIaHOK7zz77BG3Tpk3z8oEHHhi08Vrkvc+VyIG166Wc0lHY8N00XwqbxZbTLtiM4xwizhnMY/B1e//994M2nhf22bS+mPy5jz76qJdtCgTOEmx9vPg3g9ea9XeL7fe6YKuttsLQoUOLfhbPWdaK4exXaO+R8+fP97L9LN5X/D47Bt8neS557uz7+P5pf6t537Ovkp0vvqfE9hX/jtu1PHny5NT3MdL0CCGEECIX6KFHCCGEELmgQc1bMTMIhyLHzFFszrDmrbRQ9JjJidX6HPZox+OswBzaCZSX2ruU1OZ7crgzEIaVc/ikDXHmeeFQRc4aC4TZYnl9vfDCC0E/Xg9s5rFmmLRziBHLRFsqNtpoI68iZnMREF4TzgJrQ2NZXczhtDasldXo559/ftB29NFHe5n3RazAIBdHtCaW6dOne5lNktYMxuPzHNrCizzGuHHjgjY2lbIZ0GYCrs5UWyrTyOrVq/26Hj58eNDWoUMHL/N3sfcqNhnxurUmTQ4HnjVrVtDG65jD+Z955pmgX1qRUWu2SjMjW1MHr19+j70nvPHGG162+5aP2eRiQ6X/7//+D6XEOec//6STTgra7PGGwt/Z/rbyfuHrYe9Vafc4+5vJY7DckL99Nit3GtL0CCGEECIX6KFHCCGEELmg3s1bacUdbaQUZ5e0ZqtYUTsmzfRl1dI8RlohSiBU47F5y1LTbKpNgVjRTo66mTp1atDGmUO5ny04ykXnuOClVWlyxk6OCBg8eHDQjzMC8zqx0Ui81jiza4yGUPFutNFG3nTBkTFAGEXFUXCtWrUK+nHED8+LNStwRlculAiEJi02TXGkDRBGoXBWXGtKYnU7RxpZ8xYf81q0mWk5OsXO55IlS7wcK95YbUoq1T5v3ry5z5Rs55KPuRAqF4oEQjMYX0NbOJIz4dpryqYvvgZcJBgITdQcHWXv6QyPZ68vrxueIztfvM9iZmkutmmv5+mnn576vrqgWbNm3oxsrz0f87q0piT+vYr1Y+w9iOeW95Edw/7mVWPnKO13177O47Fs1xqvldj34jGsyZwLpMbI36+zEEIIIXKJHnqEEEIIkQv00COEEEKIXFDvPj1ptkBr7+TKsjbMkENt2afDZoO0WXirsbZmPid+j7WL8vtsdW+Gbf0NEb5cl6TZZIHwe8b8Gy6++GIvsz0ZCK8Ht1nbO4epcz+bLZft9xyCzdmZgbC6NIdxW3sy+/hYv5Rygn0H7FzwfollMGc/G95/tkI9hwrbNcF7lUPd7Z5L88Gxvlwcvsy+SeyzAoRzyN/L+g6wX4j1aWLfF87+y2MDa33FSpVtvVmzZv46nHjiiZneY+91/F04dNzOJV97ew/mtc8+M/YextXqeTxbwZz3La8HmyWZx+N+serbdi54zXM4v82eb9dAKbEpIuyxqB+k6RFCCCFELtBDjxBCCCFyQdmYt2xYLKtaY+F3HLZm+7FKNi301b6Psz2zuh8IQwfTVL9AqIa16v9yLEBq54S/D3/PrCG61157bXDM4eH77bdf0DZ+/Hgv87Wx4ams5ubzs0UNrSm0mjvuuCP1nDiM3qqc+bNs+HM54Zzzc2WvHadX4Pm0RSm5qCCH+8fCUC18vdgcxaHRQLiH2URtx+bxYmHJPG+8Tu364PuMzWLMZjG+J3CIvh2/XLD3Fc5yzHLWsF4hmirlt3uFEEIIIUqAHnqEEEIIkQsatOAoYyMksmaOjZmZ2CQSM2/xGBw5YKMF+H08HpsFAKBNmzZejmWMLhesWdBmJa7GRohwNt4///nPXr7++uuDfnvuuaeXOestAOy1115e5mzKNtNymukhZmp4/PHHvXzkkUcGbU899VTR99jxeP5iGZm5X0NH6H3zm98MjtlkxAU47VywaXDevHletgUhee3b7OZ8jXj/cUZtIIyEYzOyNdNwlBa/J6uJya5Z/o52f7PJLWZqFUI0XqTpEUIIIUQu0EOPEEIIIXKBHnqEEEIIkQvKxqeHw1uB0L5u/QbYh4Yzx1r7PftWsF+DzQ7L4bns02ND1nkM/izrG8E+PY2RRx55xMvf/va3vWyvG/t2MNYHYubMmV7u169f0DZt2jQvd+/e3cszZswI+qVlZrXXfsSIEV62fjxMWrZuC68hm2GW4bVRbmkJ2P+FM1jbbNZNkZiPkBAif0jTI4QQQohcoIceIYQQQuSCssnIPH/+/ODYhpMyXGiuW7duXrbFBRk2idnCkRyizWNzdmYgDJtmc4YNr2YaQ8i6zVp74YUXeplNi2wGjGFNRzwvr7zyStC2xx57eJnDpO1ncagxF1A85phjgn5HH310pnNMC8u35hA2DdlimExjmGchhMg70vQIIYQQIhfooUcIIYQQuUAPPUIIIYTIBWUTsm59KbjkQ8y3hn1/uOI6EPp+cEi8TYlv31eN9U3hc+SSF7GyA7GK1OUCl2sAwmu17bbbepmvJxBeHw5ft9+Z/WKs78vEiRO93LlzZy/3798/6MclKhYsWODl4cOHIw32JeI1A6xbWqGatLUAAO3bt09tE0IIUf5I0yOEEEKIXKCHHiGEEELkgrIxb9kQYjYlWZNDu3btvMymE2vC4PfxeLZq+6effuplNntYU0yaGctWbWeyVoNuSE4//fTg+J///KeXZ82a5WUO5wfSM17Hwr6bN28etPH75s6d62UOUQfCTNkvvPBCkW+xLjaTN5OWEsG+hzNBx0L22dQX+1whhBANR/n/IgshhBBC1AF66BFCCCFELigbPfzs2bODYzZnWFPEihUrisrWDPbhhx96edWqVV6eM2dO0G/p0qVenjp1qpf33HPPoB+bd9j0lZbdt7FgTU7PP/+8lxctWuTlu+66K+j373//28scXRWLgMqKLWb61FNPeXnIkCEbPH6PHj2Kvs7rDggzfvfs2TN1vHIrMiqEEGJdpOkRQgghRC7QQ48QQgghcoEeeoQQQgiRC+rdpycthNtm4K2srPQyh6gDYWh627ZtvWz9KhYvXlxU7tevX9CPM/cuXLjQyzZEffPNN/cy+/5w1mJLYwhZj8FZkn/xi18Ebfa4GuufxdXT2QcLCNMHsP9Mms9NXcGV5AcMGOBlu9b4/Fq3bp06nsLUhRCi/Gncv8hCCCGEEBnRQ48QQgghcoGzWYejnZ37AMDC9XYUdcl2SZK0XX+3mqG5bDA0n00HzWXTos7nU3PZYKTOZY0eeoQQQgghGisybwkhhBAiF+ihRwghhBC5oMEfepxzrZ1zUwv/ljjn3qPj1PoOzrkK59yMlLYrnXMHpbSd6ZzraF47yTn3c+fcEOfcXhv2jfKNc+5o51zinNs5Y/8Fzrk2RV5fXax/ZJwa9Y+Ms876EHEKe2emc25aYd8OqoMxxzjn+m9oH1EzNJeNn1LMIY09xDn3ZF2N1xA0eHKRJEk+BNAbAJxzvwKwOkmSP2zgmJcXe9051wzAmQBmAFhMTUMB3AjgSACrAYzfkM/POScDeKnw/y8b+Fxqw5lYd32IFJxzewIYBqBvkiSfFR5gG3cxupyiuWz8lPMcOuc2TpLki4Y+jwbX9GTBOdfTOfdq4al1mnOuOnNdM+fc7YWn2lHOueaF/nc5544ryAucc9c456ag6oe4P4D7CmM1d1UZCHsDWA7gXAA/KrTtU9AmjS585vPOua40/m3OuUnOudnOuWH1fU3KEefcFgAGA/g/ACfR60MKf8k94px70zl3nzOZHwtz8bRz7uwi417onJtYmIcrIp9/fWEtPO+ca1t4rbdzbkLhvSOcc9ukvV5YM8H6qJML07TpAKAySZLPACBJksokSRY75y4vzNkM59xfq+e7sA6uKezn2c65fQqvN3fOPeicm+WcGwHAX3vn3K2FvTYzNv9ig9FcNn7S5nCBc+4K59wU59x0V9DEO+daOOfuLMzha865owqvVzjnxhX6T3FFLCDOuQGF93R3zvVzzo11zk12zo10znUo9BnjnLvBOTcJwPn1dxkiJElSNv8A/ArAT4u8/mcA3yrIm6JqE1UA+AJA78Lr/wRwakG+C8BxBXkBgItorDEA+tNxXwD3FPt8AE8AOKMgnwXgXzT+M6h6aOwBYBGAzRr6+jX0PwDfAvC3gjweQL+CPATARwA6F67ZKwAG0/xUAHgOwOk01urC/4cA+CsAV3jvkwD2LfLZCa2RywHcVJCnAdivIF8J4Ib1vB6sD/1b75xvAWAqgNkAbqFr2or6/APAkXR9ryvIhwN4riD/GMCdBblXYW/357EANCu8v5fmSnOpfzWawwUAflCQvwfgjoL8W6z93dy68L4WADZH4TcNVb9xkwrykMI9eC8AkwF0BbAJqu73bQt9TqT5HwPgloa+LvyvUWh6UPUjealz7mJUxd+vKbw+P0mS6noQk1H141mMhyJjHwbg6ZS2PQHcX5D/gSotRjX/TJLkqyRJ3gYwD0AmH5YmzskAHizIDxaOq3k1SZJFSZJ8hapNWUFtjwH4e5Ik9xQZ85DCv9cATEHVdS5Wo+IrrJ3newEMds61BLB1kiRjC6/fDWDftNczf0vhSZJkNYD/396Zx8tVVfn+twhoGAMhAQIhE2NISIIJYJinjhEFHoMi2gjS3SD9mka0FVDp7gfYSKMt4BOxwScGjYoibUCGYEIYwhggCQmQQCYIATIQkKDREPb7o+ru+9sr9+zUvblD1T2/7+eTT1bV2XXq1Nln73Pu+q219mgA5wJYAeBXZnY2gKPN7Akzew7AMQCG0cd+W/2fx+wRqPQbQgizUXkobeLTVU/ts9X97NchP6bkqC8bn0wfAi331TgAl5jZTFQeUHqi+UHmpmqf/xppPw1F5Q/RE0IIrwDYB8BwAPdX9/NNVP7AbSJ3/+10ujympyXM7GQ0x4P8fQhhopk9AeATAO42s/NQedD4C31sPciN6ngv83XjAJzahsP0BY5KXfDIzHqjMiHub2YBlb/kgpk1LXLl+4qvvekAxpvZxFD984B3DeCqEMKPWnlIpe6PziSEsB6VCXNadZI8D5W/8MeEEF61SqxeT/pI07Xgr4MNMLPBAP4FwIEhhNVmdovbl2hH1JeNTwt9eFZ1U0t9ZQBODSHM431U+/lNACNR8bCvpc2vo9JvB6AImzjXAAAgAElEQVQS+2gA5oYQxhYcUu7+2+nUpacnhHBHCGFU9d8MMxsCYGEI4XpUvAIjNmH37wLYFgCqf/FvHirB1Mm2Ko+iOTblcwAepm2fMrPNzGwPAEMAJBdNCTkNwK0hhIEhhEEhhN0BLAJweA2f/VcAqwH8oIVt9wE4xyrxQjCz3cxspxbabVY9BgD4LIBHQgjvAFjdFGsA4EwADxa9X7X9NSAymNk+1hxjB1Ti45rGwspqv5224Sc34CFU+g1mNhzNY3w7VCbNd8xsZ1SSDkQHoL5sfAr6MFcR+j4AF1Cc1gHV93sBeL3qmT8TlT9im3gbFQfEVWZ2FCrXSF+rBFHDzLYwM/YG1hV16elpgU8DONPM1gF4AxUdcrs27usWADea2Z8BfBeVWJIm7gTwm2ow1wXVfz+peitWAPgCtX0FwJPV4/hiCIGfhMvIGQCudu/dXn2/FvfmhQD+n5n9Zwjha01vhhAmm9lQAI9Vx+UaAH8LYLn7/HsADjKzb1a3nV59/yxU+nsrVLyDX9jI+7eg+foYS1KqaJltAHzfzLZHJXbjZVRc62+jkgX3BoCnatjPD1EZay8AeAEVFzxCCLPM7FkALwJ4FRWvoOgY1JeNT1EfFiXbXAHgWgCzzWwzVP5Q/SQq8UC3m9nnUYlfTbw1IYQ3rZLAcw8q8a6nAbi+yZFQ3efcdv5t7UKpl6Ews5tRCeh6vJWfuwXAXSGE33TIgQkhhBCi3WkUT0+HEEL4+64+BiGEEEJ0DqX29AghhBCiPNRlILMQQgghRHujhx4hhBBClAI99AghhBCiFOihRwghhBCloFXZW3369AmDBg3qoEMRLbF48WKsXLnSNt6ydXRVX773Xlqcc9WqVdHefPPmy7FHjx5JO6P1Sd9/v3ih3g99qHlB4T/96U+Fn1m3bl2099lnn40ddrvx9NNPrwwh9G3v/dbj2ORznuvPRqU7jE1OZPnrX/+abPvzn5tLVG299dbR3mKLLTb5e/m7+HsAoFevXpu8/7bQEWOzXsblBx98EG0+3/7cb7XVVtHmMcrzJZBeA1tuWX/rMuf6slUPPYMGDcKMGTPa56hETYwZM6ZD9ttVffnUU2ltswkTmpfb2nHHHaO97bZpUWR+IFq5cmW0/c1zwIAB0Z45c2a0ly9PaxmuWLEi2g888EBNx94emFmuOmqbqcexyQ+0/kbG/dmR+OxUfr3ZZpvm6O7qsck3Mv9bctsYfvh45ZVXkm1z5zbXljv44IOjvcsuu2z02DbGkiXNw+D5559Pto0fPz7atT4c8+8F2ta3HTE2O3JctuY3r1mzJtrcr2wDwIgRzYsdfPjDH47266+/nrTbeeedoz1y5MjC7+Xx1pl/6OT6stR1ekTnM23atOT1nDlzos2DYtGiRUk7HrT80LPDDjsk7fjmuv3220e7T58+SbvFixfXftAigSey++67L9l22223RZsfJt98882k3dq1zQXMv/jFL0b72WefTdrxxP7CCy9Ee9990/V9b7755mjzxO0nWn7tH4gazfvEx1vrDfC8885LXv/lL81L4vFNDkj77Lrrrmvxe4HUC3DAAQdE23sR+EGXH3T8Hzj33ntvtN9+++1on3jiiUm7U09tXjKxrQ99jUzud82bl66K9O6770Z7/vz50Z49e3bSjudPnlu5H4B0/PI4GjVqVNKuHsdU97wahBBCCCEceugRQgghRCnQQ48QQgghSoFiekSn4rO3Bg8eHO233nor2rvvvnvSjjV6zrbimATfjmN6evfunbTjz3F8Tz1kWtQDHGj66U9/OtnGffjOO+8k2zjOgM85Z//4/XOcl4/lYjhwmGMUAOAzn/lMtDne4Nxzz03aXXLJJdH28QZdFXTZVmoNyr700kujvXr16mTbrrvuGm2fvcVjkPvZB7XyuT///POjPXbs2KQdB7/y9/p4O44R4mwijhcD0sDriy66KNlWxuWVFixYEO2lS5cm2wYOHBht7j8/f3If8Vzosy856YTjfXzQdkcF+28K8vQIIYQQohTooUcIIYQQpUDyluhUOF0SSOvlcFq6l8H49U477RTtXNFBlkC8u5s/99BDD0Vb8laFs88+O9peEuFUVi9bsczCEpEvLcCyJpcgOPbYY5N22223XbT/+Mc/RnubbbZJ2hVJU3fffXfSbtKkSdF+9NFHk22NIGkxubTshQsXRpvLQnjZmOUN//t5n7vttluLnwFSmenXv/51tFmaAlIZi/t1/fr1hd/LNktiAPDcc88V7oPlGN7mZZruBMtMLFMBaTmC/v37R/vWW29N2t1xxx3RPv7446N93HHHJe2GDh3a4nf5UiBctqBeihjK0yOEEEKIUqCHHiGEEEKUAslbolNhKQNIJahcVhBnArG72stWvA9213uXPMtbXr4pKzfddFO0uRqvz67h85/LGuK+8Wv38Lpo7Pb2sib3W06m4Nc9e/aMdt++6fI7LJHdfvvtyTau8NsI5JbymDJlSrS5j/i8A+m5yq1px+O0X79+yTaWqO+8885o++q8LF+z7OGvIV7XiSU8P9b5mnr44YeTbUcddVTh5xoZPh8sYQLp+eUleIBU1mSp8uWXX07a8dqFnM23bNmypB1LwyxvcgYZkEppZ5xxRovvdzby9AghhBCiFOihRwghhBClQA89QgghhCgFpYnp4VTKG2+8Mdk2bNiwaHPK7EknndTxB1YyfKwOxwewts+rMANp3A3HIXiK9HufPsvt/HeVlRtuuCHafH58OjDD8Rf+c0yu+jHj41T4uznewLfjlFyOTfGrj3Psj0/XbbSYnhx8TfO59jFTfE79uWL4vPnKzXzuuZRArh3H4/iYHh7fPF9wpW0gvaY4LR9IY3pysU+NBsfxcCwNkM5xe+65Z7KNV1M/6KCDor3LLrsk7TjlnOOk+DMA8OSTT0ab44WOOeaYpB1fN9OnT4/23nvvnbQ74IAD0FnI0yOEEEKIUqCHHiGEEEKUgu7j99sIjz/+eLT9YoVPPfVUtL///e9H+8ILL0zaXXvtta3+Xu9OvvLKK6PNacE/+tGPknZeNmhkOO2YU4aBVFpkV7uXQ7ja6GuvvRZtTtME0kqv7O71addcRdQvoChSqcPLFNyfOdkwl87O/VtUxRlIpQne5tOr+XhZHvFVYLmdrx7Labm++m+jwanDfA596QBOHfeyMY9H7qNcdXP+Lt+OpQ5u5+Unvr74e/lY/f45bb47w/MgV6b32/w4GjduXLR5juQSA74dS8tetuI+4/7nRaOBtGI7X3t+zt1rr72i7auttzfy9AghhBCiFOihRwghhBCloOHlrVoXk+PI8V69eiXbWO7iqP/rrrsuaXfmmWdGe/To0YXfxW5G3h8ArFq1KtpcHfWss85K2h155JGF+2802OW57bbbJtu4Yi67qL2kwueKXbfe5X3ooYdGm13j/tpgV353qtjaGs4555zkNZ9LPt+vvvpq0o7d4z77gzN0uA9zi1nWughk0SKSHpZl3njjjWQbVwT31+KDDz4Yba4e2wh42YolApaU+dwAqVTsFyPlMcKyYK5ysx+3DMtWtfY5Z2x56YSP11cn7k7wuOTz62VBlpL8vMhzK5/TgQMHJu24bzlji6s4A8DcuXOjXVRB27/OZVUuXbo02vvuuy86Enl6hBBCCFEK9NAjhBBCiFKghx4hhBBClIKGj+nxsQIMa8CLFi2KttcMWWvmeAVf1XLMmDHRPu2006I9YMCApN1//dd/RXvw4MHJNo6BYK19xx13LPgVjQ9XU/YxBRzbwXEJvh3HcHC1WZ9azFVKBw0aFG2fusz93J3KA7SGCy64IHk9efLkaPP59/EB3E++JAPHGXDcRm6c8rZc5WbuJ45fANL4E06j95V6+bf473rooYei3WgxPT4FmGOyeIz5Eg88R+6zzz7JNh5zuQrdvH+O1ai1CrcffzxWn3nmmWj7PufrkOMouxsch1ZUmgFIY3V69+6dbON7HI8Bf95uvvnmFvfhY+MYnit8bBnPB3yN+vmdy7copkcIIYQQoh3QQ48QQgghSkHDy1u5qq8TJ06M9vbbbx9tny7HLjhOKffVZtn9e88990Tbu/iHDh0abU7hBdIF9NgFzSl7ADB8+HB0F9jt6l3UDLtGvRueKyqz25z7FUhdvlxx18uH3Oe5NNvujF/kj69BXnzTpwoPGTIk2n7RQx4jPDa9K74o7Znd8EA6Bvkz/jpiqZjd8v3790/a8baLLroo2XbggQe2eEyNAMtAQPE1zXMOUFxNGSheFNTPuTnpsqhdLmW9qHKzl2I4VMCPbx77LHM3Ijx/su1XFuC50Pcz9xnfk/w97ne/+120udyKP4d8H8ulorOUxvLWqFGjknY5+ay9kadHCCGEEKVADz1CCCGEKAV66BFCCCFEKWj4mJ4c3/rWt6LNS0/4lb6LVgZm/dRv4xLoXtPm8vY+3Zf1atbMeRV4ABg/fjy6C3x+fOo4w3qwXyqE09SZHXbYIXnN5fd55V4fe8J965cjEMDtt99euO2zn/1stP3q1hyTw3E8Pg6kaPkY347HXC7+hK8rjk269957C35F94JTfj0cw+HjD7l0Qy7dmMemTz0vSlPPxe1wmrrfHx8HH7tfaoLjx/w+Zs6cGe1Gj+nh+Bme33xMD2/zKeE+Vq4Jf3867rjjos33ON+OxzbPpbnv5fgh34734fuy1pixWpGnRwghhBClQA89QgghhCgFDSlvsfuLXV9cdRlI0+A4vdHLVuzGzbnZuB275316qK+GWbQPduU/9thjhZ9pdPg85koM8DbvjvUp7E34qtmzZs2KNstbPjWTXca1rvgsKhSNAyCVmXKlCoqq8/q+YOkkJ7HwceRWAS/aN5CvDF3vLFiwIHnNEhFLEb78wN577x1tPzaLzmPuvPFnivrYH5+/hlim4W2+HX+vP6Z58+YVfne949PNORyDZSF/v+Mx5kt5FF3b/t7FUn/R2AOKx5u/hlgW48rSvh3Lrlw2BkjLlbQH8vQIIYQQohTooUcIIYQQpaAh5C0fOc4R/eyqu/zyy5N2ffv2jTZnKXhXXc5tzrBLj92zPvuHt/mMCP4t7MadNm1a4fc2OtxHPuuGZSeWRnxWUFHWF7vnAWD69OnRZrc+y5tAWh3Uu81FHp/9WERRhhZQvLisHy+5LB+G95+r+s3kpNZGY9myZclrlhZzlXp5LvVyVpHEV+t4qfX8+qr1LLlwdqa/Nnje9vK3X4C1kfDnna9tloH8OPTnsYha5ahcpi2fbx6Xfn6fP39+tDmr0vclj1lfnVnylhBCCCFEG9BDjxBCCCFKgR56hBBCCFEK6jamh3XCnLZ45513RvuWW25JtnE6M+ufXncsSoHPteN4Ea+lsm6eW8Gb9eqXX3452XbfffdtcNzdAa9Xs77M59THF/gUzCb222+/wu/i1EcfD8LxXo2WntzVcNqzH5tF8QI+jq7WdGh+zbENPq6EY39qjW3oTvhUdB8z0UQups7D557Pdy62irf5uY/7j8e6L0/B4zEXn8W/0Vcn9jFOjYTvO+6jomrVQLrSvE/7Lior4Mcbn28e274vebzlSkRwDBLPub7iftFK8h2BPD1CCCGEKAV66BFCCCFEKWg3eYvdmkW2h93fXmLISQ5XXXVVtK+44opo77vvvkk7druxezaXIpk73qIFD72LkN24PlW3SEpjdy/QXFnYp5g2IjmXd9FidT6VsmhR0AMPPDB5zX3B/eX7oWghPLFxuLIql4IA0pRXdpV7OapokUpPkfzpxwUfB5eCKAu+rAePuaKquEDaR7VWsvb9xd/F/eznNIbb+bHOc0Sti1T6eaWRy1D4a5t/C597L2nynJbro9y9i1/z/r3MyPdQPl5/3vm7OBXdL5DL0pzkLSGEEEKIdkAPPUIIIYQoBe0mb7X3Yn2TJk2K9te+9rVkGy8mN3LkyGjnqkuyy9u7cbkdu+NyklsukyQnnRQtVOqzYJpci43spm0il/nB2QirV68ubFeUpVWU1QWk10POda/srQpF0quHXeBewuCFXLlvvBu9SEbOucdzMim/zskqtf7GRsBnPTEsEbCkNWrUqKQd95GXHIoq3+ckEc7qKcogA9L5zo9N/l0777xztL3Ewr8rtzg0HwcfX73iJUi+tnl85GT5XAV0nhe9ZMjkxjlnFfP+/Lhk2Yrvs/4a4v2/+uqrhcfUHsjTI4QQQohSoIceIYQQQpQCPfQIIYQQohR0eEVmXxnyD3/4Q7RnzpwZ7bvuuitpN2fOnGj7lbQ5TZm1Sp+2yXplLhWdKUpL97C+7LV11lP9PviY+Lu8/t3UrtHjDoB8H/EKurwysj+nu+++e4v79qnsRZVCc2UFcrq22JCiGAMgjSXhvsilVPM+/Djg8cN95vuTr5futHp6Do6B8/A5LYq/APJxN9w2d05rnVuLUqV9HAiPR67o62NYeAVvH6vE+1y+fHm0d9ttt5qOtSvxfcK/hX+zHwO77LJLtPn+CaQxrbmU8KJ+9nMkV8DmlQVmzJiRtOPKyxyf5ePH+BryMU3tTTlmByGEEEKUHj30CCGEEKIUtFnemjZtWvL68ssvjzannLFrEQB23XXXaK9ZsybaPh3x8MMPj7aXeNjdx9tyLjj+jG/H1VzZtejdh5xmmasoy2mg3v1fVImUzwUAjB07FgDwi1/8At2JFStWJK+LZELv8ubFY3OwG5f350sCsIu3jBV8W6LWdO7c4oA8tlje8tc37z9XlqFIbvbfy9t8pdqi72103n777Wj788HzE1fMHThwYNKOx4iX4nkfOQmrqGKwx6dRF32Gxz6nzQ8fPjxpx/cZP6fzMbFE1gj4tPqiMiecDu63+arORXOcPzd8vnnM+oWv+Xzz/W7RokVJOy41ctBBB0X73nvvTdrtv//+0fbX2osvvhhtv+pCW5CnRwghhBClQA89QgghhCgFrZK31q1bF6Ouzz///GQbu7s4I4dtIHWhcmS3d0/mFjtj2AWby9DJwTITf5d3u7KLkGUwzjryx+EXN2W3Y05+OeKIIwAUL7TZSHA/+CyepUuXRjuXzeYz+Ipgly+7//15bO8K4mWCJRKWkIG0siqfV9+fvK0okwtI54tcBWK+dmpdOLPRyUn2RfPMxz72saTd7Nmzo+1lFZ7HctXNef/8Gd+X/Dnen5fm+Dj4N+61115Ju9tuuy3aXj4tygBrBPwcyfMnn+vDDjssaVd0HwOKJWQvafK4zI0j3j/Ps76PGH4W8NIc95efj9s7m0ueHiGEEEKUAj30CCGEEKIU6KFHCCGEEKWgVTE9K1aswA033ABgw5Rijs+pteIjp4p73ZV1TL+NNT/WJH01SY6T4f3l0ju56qf/jZwi+cYbb0SbK2ECQL9+/aLttUuOLeFjYl0UaNZMu3t12SK93act9u7du6b99e/fP9ovvPBCtP0qwaxXN8LKy51BUQyH7wuOF/ExAXwuc6noRSnQfszxGOE+8/F6uZiTWo+h0WK7chXj+bdxOx9jyLFWfozVGtPD8R3czsdg+b5tws+RvA+ec30MC6dK+5gxjr/06db1jo/P4t/C81guBisH3//4vu2/m2OL+F4NAK+99lqL3ztkyJDCdn379o22j8Hia8NX38/F9LaF7n1HFUIIIYSoooceIYQQQpSCVslbZhZdpV6WYFmI3W5eSmLXJUtEOVezlybYRcv78+69orRILxmxG5bdcd4tetRRR0X7iiuuiPZ9992XtOPfkquuyS6+jl5krV7wfcRSCV9T/rzxonY5dtppp2hzJU8vH/LrRliEsCvxMhVf334s1Soz5RaDZYq2eWmHr53uUOahFnIyI8+ZPL/l5C2ej4F0zLHU4Ste85jjbV6m4X7hhahfeeWVpB3LVjxHevmRj5cr+gLp7/cp4PWOvxfyWGGZyVdZ5jHg5V8eR0WLMvvXuQV+uR33l5c0uQI/S1hcnRlIr2VfvqW9x7M8PUIIIYQoBXroEUIIIUQpaJW81a9fP1x22WUANlw4curUqdFmt6OPDmc3GbvnvHuW5ajcQnhs+3ZF0he7Vn27L3/5y9H+0pe+hFq49dZbk9ecveXdguxeZtdyUWZDdyPndmUXp88W8K7yIjgThD/jrw0+37ksGJHPdvRySVG2laeocq+XMLgd789/b1sq8DZ69hZfw15yeuedd6KdW9iYf3OuMnLRopdAei9gSfmjH/1o0q5IBvPyKVf55mP3WbL82i9E+dJLLxUeb73j50g+Pywf+dUOZsyYUdP+eez4c8/jiMeHD/Vg+dBfUwzf41nG3GeffZJ2Dz30UIvHB2wYmrCpyNMjhBBCiFKghx4hhBBClAI99AghhBCiFLQ5mOH6669PXnN8yrXXXhvtCRMmJO04JXz16tXR9lUXOU3Nx3NwSht/r0+X4+/iz3zzm99M2n3961/HpsArFQOpdun1WY5b4QqVTavXN9GkQxdVrm0kOFbAp1ny7+PU0l133bVN3zVo0KBos5bvyx4wiumpUHSttWaV6qIV0328TFFqe26VdSYXi8BjrDvDsRS5uAo+v0888USyjeNCli5dmmzjc8r7933CfcH782Od98Gf8RWZ58yZE21Om7///vuTdjzf+5gmjgvxc2sj49O5GZ7jcqno3H/+/lQUk+dLiPBczePNx/BybCbfqznNHchXb/cxPpuKPD1CCCGEKAV66BFCCCFEKWizX9+nYrP766tf/WqLtofT3J955plkG7s4lyxZkmzjFDZ293k32D/90z9F+5JLLik8jiJyFZ6Zb3/728lrrk6dWzyOXXyjR49ucd+NlkbbEuzW9O5UlqDYXe3dn7XCabF87vx55O/1xyRSOP0ZqD3FnG0vnRUt8urd8uyK5+/NucP94pPdleXLl0d7zz33TLbxHMkp4D7tm6VnP3+yhMH95fuySL7OjXXe5stTsJzKko1PPefvmjdvXrKNr5tGn0N5XhwwYEC0fRr5888/H21fobpIdvbjjbdxn/vwAJYMi1ZI8Pvg35ELKcitYtAeyNMjhBBCiFKghx4hhBBClAI99AghhBCiFLQ5pqcovqU1HHPMMS3a9UKtv/Gss87q4CNpbDjGoiiWA0h1Z46LyrXzej1rzzmtmeMIcunsZaLWlPXc+S8aM7mV1HOaPcdx5K6jolii7kxRPByQXvsrV66Mtu8vjon0KeY8LnKlMzh+aPDgwYXtisa37y8u5cHXkz++XPwQ//5GK0nBMVgA8Oqrr0Z71KhR0faxrosXL472yJEjk208xvh8+HPP55HLhvilm7gd96WPM+JtHIPmr0M+Jr/EVXvHXMrTI4QQQohSoIceIYQQQpSCxvL7iYaHK6x62BWaqzzKLlnv+uTqruwy9bILu1clb+Xx8latKeFcriEnYXHarO8L7utcP3H/slu+0VdSz8FV7L0kwpXJueSAlw64SrKXlLktn19fPZ9lJpbZOOXdw8fr2/F3cX9xpXsglTi93MnzTE5yq0eGDx+evObj54rHXnI66aSTou2rkvM44HnRjw+WBXn8+rIVvGICzw9+PuZ5nGVWX37glFNOiba/lnMhEW1Bnh4hhBBClAI99AghhBCiFEjeEh0Ou8k5gh9IFyjkyq45KSMnbxVVAPWyBks0ucUay0SR9OPPD7vE2WUNAMuWLYs2u+J9lgjvg+UtL0OyLMbXjt8fSwBczZ0zi4C8vNpoDBs2LNpemuJFkL/1rW9F22cysUTCYxFIZaeXXnop2pMmTUrasZTG/Td//vykHZ977vNx48Yl7bhvuf/88bHkMmPGjGQbV3Q/9NBD0Uj4CtX+dRN+FQMmt0hnbgFh7j+Wmfw8y/vgedtTtMislyq5ojhLZx2BPD1CCCGEKAV66BFCCCFEKdBDjxBCCCFKgWJ6RIfDK/6ecMIJyTbW9nv37h3to48+unB/uUrZvIo068Q+toOrvnJsRJkpqlw7fvz45PV9990Xba4CC6QxPqz1+7ggjhfg9FXftxx7xTFCfrVwTpseMmRItHMxPI2evs6pzRdffHGy7ZFHHon2iSeeGG1OQ24rl1122Sbvoz3gmJ4LL7ww2XbYYYdFu9EqMufg+dLH7XAcpI+zKSoB4tPBebzx/vw55DhNnkt9vBDHI/ExFMUpARvG67XH6g/J/tp1b0IIIYQQdYoeeoQQQghRCiy3kNwGjc1WAFiy0YaiPRkYQui78WatQ33ZZag/uw/qy+5Fu/en+rLLKOzLVj30CCGEEEI0KpK3hBBCCFEK9NAjhBBCiFJQFw89Zva/zCyY2b41tl9sZn1aeL9V6wm0tn1mP2eb2a4bb1luzGxHM5tZ/feGmb1Grzc9l1a0K23tLzMbZGZzCrZdbmbHFWzbYByZ2WfM7BtmdpSZHbJpv0i0lWofzDWz2dX+PzgzD59oZpcU7Ef92MWY2S5m9kszW2BmT5vZ3Wa2dyv3sb2Z/WNHHWNHUi8FDM4A8Ej1/3/r4mNpC2cDmANg2UbalZoQwioAowDAzP4dwJoQwneatpvZ5iGE9ws+3u6YWY8QwvqNtywnG+uvNu7zX1t638x6oOVx9HEA1wM4AcAaAI9uyveL1mNmYwF8EsBHQgh/qT7oFD70hhAmAZjk3zezzQEcBfVjl2GV4lR3APhpCOEz1fdGAtgZwPzcZx3bA/hHADe0+0F2MF3u6TGzbQAcBuDvAHyG3j/KzKaZ2W/M7EUz+7m5amJmtqWZ3WNm/9DCfr9qZk9V/zL5P5nv/171L5gpZta3+t4oM3u8+tk7zGyHovfN7DQAYwD8vPoXUMtVoESLmNktZnajmT0B4D8z536amY2p2n3MbHHVHmZmT1bP/Wwz26v6/t/S+z+q3lRhZmvM7LtmNgvA2C750d2IovMPoIeZ3VQdW5ObxkW1v0+r2ovN7GozewaVP3iScVQd76MAvAXgiwAuqm47vOpNmlr9zilmNoD2f6tz8W4AACAASURBVKOZzTCz+Wb2yc4+J92QfgBWhhD+AgAhhJUhhKYH0wvM7Bkze86qnvqqx+7/Vm0e37fB9WMX/JayczSAdSGEG5veCCHMAvCImV1jZnOqfXk6ULk/V8dXUx+fVP3YtwHsUe3Hazr/Z7SdLn/oAXASgHtDCPMBrDKz0bTtAABfArAfgCEAeLncbQDcCeAXIYSbeIdmNg7AXgAOQmXSHG1mR7Tw3VsDmBFCGAbgQTR7mSYAuDiEMALAc7n3Qwi/ATADwOdCCKNCCH+GaC39ARwSQvgyis99EV8EcF0IYRQqN82lZjYUwOkADq2+vx7A56rttwbwRAhhZAjhkRb3KFrDBue/+v5eAH5QHVtvAzi14POrQggfCSH8DBuOowMAzAohLAJwI4DvVbc9DOD7qPy1OgLAz1HxBjUxCJWx/wkAN5pZT4hNYTKA3asPkTeY2ZG0bWUI4SMAfgjgXwo+3zS+T8GG/Sg6l+EAnm7h/VNQuVeOBHAcgGvMrB+AtQBOrvbx0QC+W/1j5BIAC6r9+NXOOfT2oR4ees4A8Muq/cvq6yaeDCEsDSF8AGAmKpNZE78D8JMQwoQW9jmu+u9ZAM8A2BeVSdjzAYBfVe2fATjMzHoB2D6E8GD1/Z8COKLo/Zp/pcjx6xDC+jae48cAfN3MLkalNsOfARwLYDSAp8xsZvV109oE6wHc3u6/oLy0dP4BYFEIYWbVfhrp2GV+VfA+AIwHcE/BtrEAJlbtW1HxFjdxWwjhgxDCSwAWojL+RRsJIaxBZTydC2AFgF+Z2dnVzb+t/p/r419LRq57DkPFgbA+hPAmKk6AAwEYgP8ws9kA/gBgN1SksIalS2N6zKw3gGMA7G9mAUAPAMHMmp4c/0LN1yM93ukAxpvZxLBhsSEDcFUI4UetPCQVLeoa3tt4E7yP5of0+Jd7CGFi1XX+CQB3m9l5qPT/T0MIl7awn7WagNuOmZ2MZu/b3xec/4XYcOwWyb65vh+HYg9RDj+ONa43keqYmQZgmpk9B+Cs6qamfvbzM1PL+Badw1wAp7Wi/ecA9AUwOoSwrhpW0NCe06729JwG4NYQwsAQwqAQwu4AFgGoRev9VwCrAfyghW33ATjHKvFCMLPdzGynFtpthuYL4LMAHgkhvANgNenNZwJ4sOj9qv0ugG1rOGaRYSPneDEqf20CNGjNbAiAhSGE61Hx/o0AMAXAaU19bma9zWxgx/+C7k8I4Y6qS3tUCGFGwflvK3EcVb1+m1eDqZNtVR5Fcwzg5wCwVPIpM9vMzPZAxcM3bxOOqfSY2T4UqwVUZJC2VhnWXNm1TAXwYTM7t+kNMxuBigR9upn1sEps6xEAngTQC8Dy6gPP0QCa5tGG7ceufug5A5VIcuZ2pBJXjgsBbGlm/8lvhhAmo+L6fqz6V8lv0HIHvQfgIKuk1x4D4PLq+2ehomnORmWAb+z9W1CJHVAg86ZTdI6/A+B8M3sWAKfJfhrAnKqMNRzAhBDC8wC+CWBydT/3oxKMKdqfDc7/JuzrFlTHEYATUXGnN3EngJMpAPYCAF+o9u+ZqMwFTbyCyoR9D4AvhhDSJadFa9kGwE/N7Pnq+d4PwL+3cV++H0UnUlVFTgZwnFVS1ucCuAqV++VsALNQeTD6WgjhDVTi5cZU76OfB/BidT+rAEyvBj43VCCzlqEQQtQdZnYzgJtDCI+38nO3ALirmmAghBAJ9VKnRwghIiGEv+/qYxBCdD/k6RFCCCFEKejqmB4hhBBCiE5BDz1CCCGEKAV66BFCCCFEKdBDjxBCCCFKQauyt/r06RMGDRrUQYdSzLvvvpu8/stfmou99unTxzdvN1asWJG83nLL5hI822yzTYd9L7N48WKsXLnSNt6ydXRmX37wwQfR3myz+njO5gB+s3Y/vYU8/fTTK0MIfdt7v101Nmtl3bp1yeu333472uvXNxfI9okV227bXF6rs8ZcrXSHsSma6YixWS99+dZbb0X7j3/8Y7Tff//9pB2PPx6Xm2+ePirwWNxll13a7Tjbi1xftuqhZ9CgQZgxY8YmHUxbbjYPPPBA8nrhwoXR/ru/+7tNOp4cN9xwQ/J6xIjmYrOHHXaYb94hjBkzpkP22x59WSt//nPzGqz84NiV8GD3A7ojMbO2VrLN0pH92ZoMz6Ix/dprryWv77rrrmivXr062v7h6Oijj452bswVzSv+2NvzAbc7jE3RTEeMzXrpy4kTJ0Z7ypQp0V65cmXSjscfPxx558Khhzav/f3Vr9bfeqO5vqyPP7uFEEIIITqYuilOyH/tAcCpp55auG2LLbaI9uzZs6PN7jgglVJYYmFXn+eNN96I9vLlywv317Nn85prTz75ZOH+ROrd+etf/5ps4/O92267RTvnXWDP0dq1awu3rVq1Ktq9e/dO2g0cqKW42oOc54S9Of/93/+dbOP+6Nu32QvN4xRIva3z58+P9jnnnFPzcTBdJWsK0R7UGiqwww47JK/feeedaPfq1SvaXpp6773mtWG33nrraC9YsCBpN3ny5Ghfdtll0fbzMVMvY0+eHiGEEEKUAj30CCGEEKIU6KFHCCGEEKWg02N6irS8iy66KHn94osvRnuvvfZKtvXo0SPaTz31VLR33333pB2nun/84x+P9mOPPZa045iTNWvWRJvTZf33vvTSS9G+5ZZbknZnn302RMucd955yet777032ttvv320fUzPhz/84WhzhoGPAeHri/vft1u2bFlrDrvU+DHL59Jvu+OOO6I9YcKEaPusLI5H4DiCHXfcMWm3xx57RHvq1KnRHj16dNJu5MiRLR5fvZRIEKI9yF3PL7/8crT9fMfjhctF7LzzzoX75xhZjmEF0pjIxYsXR/vSSy9N2l111VXR5rnCH19njlPNCEIIIYQoBXroEUIIIUQp6NKUdXZxzZs3L9nG7jNfGZlTXNkFxymtQJpyN23atMJ2RcXpvMuN06379esXbXbhAZK3csyZMyd5XVTNk6tuA8Drr78ebZYgfer5dtttF212ydZLUcRGxEuNOVc0p6lzyQDuPwAYPHhwtDnN9cEHH0zacRkDliSvv/76pN0Pf/jDaH/oQx+Kdle60TeFpnPemam9uUKOuXRjnoP5/Pp2bSkgWS9pzp1JrQU1Fy1alLzm1HGeB4G0OCgXZuUSH0B6j/vTn/4UbR86wvvg9Ph77rknacfp8Zdcckm0/TjsTEm6MWYAIYQQQohNRA89QgghhCgFXSpvXXzxxdH2cga7qDlzB0izqFi28K46XjuEJRHvPuTXW221VbR9hWd2w/MxsIwGALfffnu0ubK0SCswA2llXj6PXvZi9+yQIUOi7WUrvm7Ynj59ehuPWLRGVth3332jzZXT/Tgoqm7Oa20BqbudK7N7mZQrzuYqPDeKvFV0zp977rlo8/nl+Q1o27pguX7ObeO5sC37b+v3dldyv5krkd9///3JNl4fy6+V9eabb0abwzn8gqMsJ/Mal/764nshz9t+UWCuxP74449H+3/+53+SdkWrJ/ht7UFjzABCCCGEEJuIHnqEEEIIUQr00COEEEKIUtDpMT2s13FlZNbkgVSX9zE9DMfj+NgaHz/S0jEAwK677tri/nyMEH+ONU3f7gc/+EG0FdOT4ldZ53gAjuvieBwgrRzKn/GadFGsiNfJlyxZEm2tuN5+vPDCC9F+6623or3nnnsm7ebOnRttjgPysX2cNstjzldL5/i9XExPI6RAf/DBB/F333bbbcm2SZMmRXvEiBHR9nEPDz30ULQHDBgQba7GC6TnzVe+51IhfE49vE+eq/0xcYwk75srsQNpn+Xmfu4/P6/wvMDXlC9/wjEy9coDDzwQ7UceeSTavr/4vHG8F5DeG3lu9WOAq9gfeuihLb4PAEuXLo02xwj5ccnzNs8NV1xxRdKO0+2Vsi6EEEII0Q7ooUcIIYQQpaDT5S12XbGr7vOf/3zSjhcSzbk/2WXqKytzOjSnu3I1Zf85XvzQu9nYvc7782m23iVddvi8LV++PNnGrneWrfwCleye5TR17/72qZVN+IUsubqv5K0KLP2wnXM3//jHP05e9+/fP9rDhg2LtpeZeAyy69zLleza32+//QqPiVNgv/KVr0Tby6S5xVLrhXfeeQd33nknAGDmzJnJtiuvvDLaDz/8cLR54V4glXZHjRoVbV/Fl2UQvxAzpz1zyvPKlSuTdlzmg2UwXjQaSMcgt+M0fCAd3zz3+7HOEh5X/wbS38zyKc/vQLpwdL1y6623RpvvVV7SY/y1zeeO51l/Tvl+yteGL0vwhS98IdqvvvpqtP1qByxPc+Vmlro6G3l6hBBCCFEK9NAjhBBCiFLQpRWZmQkTJiSvOetpypQpyTZ2XXLmVG4RM3atetcfSyIsxXi5jDMdLr300mh/+ctfhiiGs3j8OWWXp88QYIqyONiND6R9xN/lKzz7bEGRjouiRSQBYOrUqdF++umnk20sTfD59/vgBRG5L1iSBoATTjihxW2cPeJfX3jhhdG+7rrrknZ8HLUu7NjZbLHFFjGj1MsKM2bMiPaTTz4ZbV7Y0b9mGejII49M2nGlcz8Hjx8/PtqLFy+Otj+m008/PdosX7O0AaTzAG/zUschhxwSbZ63vXTCIQZ+XuHrizO2WBIEUpmmXmGpn8eln8P22GOPaOfmUsbLyfyav8uPDZYu+TMsgwJpWALLZSyJdTby9AghhBCiFOihRwghhBClQA89QgghhCgFXRrTwzE3XvPnlcpZTwaAAw88MNqsY/pqrqzZsz6Zq9LKPP/888lr1kk5TVPkYS3fr4ruU9Ob8CvcM7mquryNv8tX6/ZptyIlt3L2o48+Gm1fToJjrzheZPjw4Um7efPmtbjNlxzgOABOofap15wCz3FdfO0BaVyQnwdqXS28o1m7dm08P3wOgTQWgs/bggULknY8Z86ePTvavrwGV633VbM5DZxXz+YyEx4uEbD77rsn23g+5d/lK9ozXNG3KY2/pW3++nr55ZejzeVPfKxL7rvrBZ6r+D7p42d4ZQEfA8lxN3yd+3tf0X3Sl37g65C3+YrMXHl9n332ibY/71w6wFeabm/k6RFCCCFEKdBDjxBCCCFKQafLW0WVXr2cwS44dmsDqQu8qIosUFx91bu1+bt5H76dJK32h0sE+EXyGJYu2VXr+4T7L7cwaa6aaVmpdTFOlo/Y9rAkwlIEALzyyivR5vRl/73s2ucUZS+H83Fw3/qKxsccc0y061Xe2nzzzaMM5yuYc+kFlrT8b+HPFX0GSCtZjxkzJtnGEsbIkSOjzSULgFRq3H///aPNshKQpqJPmzYt2l4ifeaZZ6LNfeLvESzh+YVEWT7h/ft7RJG8Xk8UpZ/7OYylSn/PZAkqFzrAIQFF6et+f2x72Yrndx7b/D6Qyp2St4QQQggh2gE99AghhBCiFOihRwghhBCloNNjeopiBXIxBEVLEACpJutT1nmJgqL09dz+fGnzIuq1nH29wNqzj8Xgc8wxIF7zZV2eUx+5FD+Qlp/nfvDfWy/xG/UEx4Xw+fHxEhyDM2jQoGQba/ODBw+Oto/v4L55/fXXo80xIUAaV8JLEvgYLU6N5RgWv4I3x/TU6zhdv359XA2czyEAHH744dHmldV9LMXQoUOjzWPCpzl/6UtfiraP1eF4Kl4K6NBDDy08Ju7/448/Pmk3a9asaPPSE2eccUbSrmj5C44rAoDHH3882r40AbPffvtFm1dcBzaMNatHuLwDr07v73eMvydxW77H+THA82Qu7pHHX1Ecpd9/UWkYIB2nRx11VGG79kCeHiGEEEKUAj30CCGEEKIU1M0q6zlXs09l5hQ5drPlUp7ZVefdbCyxsItfKertA5cY8JU9mVyKOUuc3Ed+JWeWwfh68PJWTuIsK0Xu50mTJiWv2cXOUiOQjiV2qbPEAKQp1Xx9eJmCxyDL1T6Nt0kOAlI5h9N4PbXK153N+++/H2UolvSANAWf0/T93McrcPM5YIkJAI499tjCfbCs8p3vfCfafl689dZbo83yll/BnGWLBx54INr+GmKp7je/+U2033777aQdV5D2cviyZcta3J+/Dmtdjbwz8WOAxwdXXfbyFs9pPB6A9Pzw+PDnjffBc6afjxmWy7wkxvvge7y/3z/99NOF+29v5OkRQgghRCnQQ48QQgghSkGX+ndrrQDrYXcou3G925VdciyJ5Ko/87ZevXrVfEyiGHahekmB3Z85eYsrjLKL11NUYdV/r5fFRPEY9NlbPG65si6Q9ufAgQOj7aUJllx4kUKfbcVyJR+flwB4rPLisn4BU5YEclmhXclWW22F0aNHA0grJgOppMOLrD744INJO5YPOUPLZ29dffXV0fbn45prrok2Z8Rdd911STvO8mL5+rHHHkvanXDCCdH+53/+52j7a4ivDc7Y8jIYL0DKWX5AugApSy5e3vvoRz+KeoOrlQPFKwt4eO7zUiXPrTlZl8dvbnWCos94+Lty2Vv+N3ck8vQIIYQQohTooUcIIYQQpUAPPUIIIYQoBV26ynpbK6JymiFrlV4zZH2ZtX2OIQCKV+32WiWv8rzDDjsUfm+9VnrtKmpd0Zx16Fxf8rnnVYE74pjKRFGV6jlz5iSvP/KRj0Tbx4HMnz8/2txn/fv3T9rxGOG4Da7K7dl9992jvXTp0mQbx43x7/Bj+KWXXoo2x33UE5tttlmMS7rnnnuSbcOGDYs2VzJetWpV0o5f83mbOHFi0o7T3pcsWZJs43iXPfbYI9pnnnlm0u63v/1ttDn2g68TIF2NnWOreF4F0muDf8cBBxyQtONtfh8f//jHo/2Tn/wk2j5FOxdn0lX4uCueF3MVjnMp4TwOOG7Vx7cWnQ+/Pz6PfHw8NwNpfBaXDvD7y5UyaW/k6RFCCCFEKdBDjxBCCCFKQd0sOOpT4tgd9+Mf/zjZxi45Tmn1i+7xPtj2KXuc6sfylq/meumll0b7xhtvbHHfYkO4v3KL5PG14eUndqGypOJT2/m7WObwqey54xCpXOAlJ3a/+xRzlqo4zXnhwoVJO3ajc/kAvwAkp8uzPOJT0bnfX3zxxWj7sckLn9arvLV27dpYDdlLRPx7nn/++Wjzop9Aer1Pnz492iNGjEjacXVeXgQUAAYMGBDtn/3sZ9HmSs1AmorO/fLII48k7XgMjxo1KtpeouaK3zwf//73v0/a7b333tG+6KKLkm0ss/K14e8/XiatB3yJiFw1ZKZIBgOK50U/PmoNzeB7KO/bl41hGSwX2sKlZzoa3a2FEEIIUQr00COEEEKIUlA3K+7l3GpTpkxJXhdVUPawa42jw73UwdIa21zZFejcRdG6E9xHXsZklye7Wr38xFkBLJvkZLBcZkZR5WZRgc8rZ/gAwLhx46LNlX+BtN84Y4tlaCCVyF5++eVo++warvbLFZ69lM3zBy8q6bOacguQ1gs9e/bEXnvtBWDD38nXPlco5kU/gfQcDB06NNpXXnll0m7s2LHR9ufm7rvvjjZLLr76MUtavCjsz3/+86TdSSed1OJ3+Wq8LLm9/vrr0T7xxBOTdnyt3XHHHcm2gw8+ONpN1a2BDStcs0RWL/hMNO5zxmdKcbtas9T8fMz31tw9mbfxPvy8fdBBB0Wbq6j7edtXbO9I5OkRQgghRCnQQ48QQgghSoEeeoQQQghRChoipsdXqOS2HC/iU9FZx2QN0VeR5f3lNE2/cm0RrHEqnT3Fn0M+x3yufErybrvtFm1eadprw7yP9957r/A4ak0DLSu33357tH3KOp9zf46feOKJaHM1Yd+O40K4FMSvfvWrpB2nM3NMnU9xPe6446LNFdtfe+21pB3HBdUrIYQYc+ZT0TlW44EHHoj2jBkzkna77rprtDnOZsiQIUk7n37O8Ng85phjou1jvDjeh+fW/fffP2nH8R0cq+TjQDiOi+d3riwNpNW1fUwPH9PJJ58cbR8X5NPD6wEfx8Xnh/ukV69eSTtO9ff9yqnkfH/ysT5FMZa5Cs98z/TH3hSbBqTXjY856sz5WHdkIYQQQpQCPfQIIYQQohR0qbxV6+KjnLYIpDIWu8l8inlRJU4vOfFxFFWuBFL3nCSs2ilyzwJpX3JZAe/uZHf9TjvtFG0vm7B8xv3nZTWlrOfhKsle3uIFSPv165dse/bZZ6PNfe0rtbLkwqm3vp/YXc5j07vlOe2dqzp7iYUlkXpl3bp1cc7j9G0gnWu4DID/nfy5CRMmRNuHCvTu3TvavjIyV3LmscTp4ECa9s39dcEFFyTtWJ7MLSTKktPixYujPXXq1KQdLyrqK1dzCjTP1V4iq8cFR3lsAOl1z/Pivvvum7Tbcccdo+3DA1gKy1WoLrqv+XtckfTl51WeH7gaui81k9tHrWEltaK7tRBCCCFKgR56hBBCCFEKGkLe8hJGkavOZ28VfZeHvzt3HOzy5+wRXxlTpLC8lcsW4L702TnbbrtttFne8q7QomvKy2Xcl2JD+Pz4DDmWlHlxTyCVQXJjjscqt8tV7M6NTc74YQnDZxp5t3890qNHjyhP+QUxuZLxmDFjos3yLwAsWLCgxW2DBg1K2rF85LNajz766GjzNeBlFa60y3KZl9J4HyzFLFmyJGnH+2Cp0lftZfmNq1MDwPHHHx9tXnyUrxMA+MQnPoF6w1/nPMfxNl/lvKhKMpCOt1xoRm6FA6ZoAW9/r+Z+5uuLMyyBVNJbtmxZsq29My7l6RFCCCFEKdBDjxBCCCFKgR56hBBCCFEK6qYicw6uxgukeiDriV4L5XgAtn18B38uF0PA2irr2IrpycPn1MfgFFXi9LEXPhahCZ/Sy/EmRVVIgdq167LCuvohhxySbOMU0ueeey7Zxv2bG5tM0TgF0n5j25eT4O/ldGhOkwbSmAMff+BLXnQlTTETvlrxY489Fm1Ov/fXN8e/cEViP44effTRaPu0d37Nx3HTTTcl7fh66NOnT7T9GB4/fny0OR7p6quvTtrNnTs32v/wD/8Q7ZEjRybtrrrqqmj7siZ8j+C4KK4QDGwY81UP+NhU7luet3y5CJ5Lc6VBeKz4cVT0vbmUdbZ9RWa+Nw4dOjTaXK0dSMsl+FXmFdMjhBBCCNEG9NAjhBBCiFJQNynrHnbjeZdZUSqyd+nlUpZr+V7v+uPjZXfqHnvsUdO+xYayEvcLu9C9i9cvlNgEp7cCqUvdp3SKPFwmgM+jH6ecDu1TgNtCTt5i2N3uq7SyTMHzBS9ECgCTJ0+Otpdf6kXe2mKLLWKqtq+SzBIBjxefzs0p20ceeWS0uWI2AIwdOzbafoxx2QL+Li+RcWo6n1MvzXGlZa7qPWzYsKQdpznzvhctWpS043nXy3t8PfB9wFcX5++qF7gyPZAeP59TH/bBcqffR1EFZS9bFX1XbvFt3keu0jJfNz7Mgffhy5W0N/L0CCGEEKIU6KFHCCGEEKWgS+WtXEYHZ+HkqviyW7PWxeNy7Xibd/3xd3nJTRTDrlAvMxZV6fTyVpH04CUsdq+zqzXnThUVWH5g1/m8efOSdtyHPoOEKzRz5XRPURX0WrNEfOYVVyrmY+jbt2/Sjl32zz//fLKNq/92JWvXro3n/Je//GWyjasrc5VyzpoCgIkTJ0ab5UifocWSka/+PG7cuGizLMbZccCGklETPguHF4VlWYmztYB0rHO7mTNnJu1mz54dbZ/FydcHzyV+wdnHH3+8xWPvSvzcx+ODq1r7xVP5/HhZlO9duftu7jgYnlt5fvff6ysvt3Q8nvaQzHNo5hdCCCFEKdBDjxBCCCFKgR56hBBCCFEK6rYic66aa1FaeS72h8lVZM5pnxxTwKvCijxcGdn3CafF8vnmeAWguHJoLqaEdX3/vTm9uqxwrMarr74abZ/KzFVt77jjjmQbx2jxOM3FEXA7r/Xz5zgt25eJ4GPia8fHGHD8Qa0xgJ3NZpttFn8Dx9UAaawjp337FdIPPvjgFrfxeAPS1G5fBoCrWXPsXG6lej73PhWd511fQZnhNHVeBd6nQw8YMCDaPs6IU7Y5Vdqn2/vV2esBn+rP8Dnwfc7bcvMbz6X+XshjgtvlVjtg/Hgr2l8utjN3fbUH8vQIIYQQohTooUcIIYQQpaBuffzs7vKuOnbx1pp+x9T6mZz726dI1vq5sjN48ODkNaeScxmAogrMHl+VlNNfuZ/9NSR5ckM4ZZ3lDJYbgLSfvDs7V8mZyaWsMuwS58+cffbZSbtPfvKT0f6bv/mbaLME4qm1Sntn88EHH0TZyafc83j5wx/+EO0DDjggaXfQQQdFm9PZH3744aQdlxXw0hennPOipX4R11deeSXaHALA6fVAKn2xfOplGv6NfB369GeWpnx5BF7Q8thjj402p3wDqXxWL/hyDCw78jYu0wDUXlG81groRWUlcvvwEilfQzyWfZ+zHMn3945Anh4hhBBClAI99AghhBCiFOihRwghhBCloG5jehiv//EqrG1ZTsDrmKw1ctqfT5Hk7/Jl35m2xBl1Z7jUvU8t5VXSOSX5kEMOqWnfPmaD+4y1YR8PUI9aflfDcRF8Xr3Gzv3kz2uty0vstNNO0V62bFm0c8uK8Jj73ve+l7T7xje+Ee2RI0dGe88990zacRxMR6/m3FZ69uyJ/fbbD8CG8R0cm/apT30q2n6u4iU2uKyDL/HA5+quu+5KtnE8Ecd1+XjG4cOHR5uXjfBLv/B1xLF4/pj4u3hu9tcGxwXx9QSkq9Hz8hp+pfbTTz8d9Ya/P3EsFMdP+T7nmB6/NAiPv6LyH0AaN1e0MntLr5vw/cAlEbhPal1JviOQp0cIIYQQpUAPPUIIIYQoBQ0hb7H725Or9ltErWl63iXPrmX+3tbsv4xwaqlPWd9ll12ivXDhwmiPGjWqpn2PGDEieb3DDjtEm+Ua7wr+2Mc+VtP+ywSnorNb2q+WzbKQlxfZ/c4yYdZ6kwAAB29JREFUmD//nDr81ltvRdvLn/zdPP68e7wofdmvEM+p7bWm+HY2W265ZVwN3a+K3pF8/vOf77TvErXD8hbLT74q+eTJk6PtpVsOEeFSDX5cMrWGaeQqLfOcfuSRR0bblxDhz/myAu2NPD1CCCGEKAV66BFCCCFEKehSeatW9xlnBAAbVqJswi9Uxq85ItxHhxctzuarzeZcgYyyt1JYUmC7PWCXKQBMmzYt2rksBbEh7ALnqrucYQcA/fv3j/bEiRML9zdr1qxoe4maZSxemPKEE05I2vGYyy1myVla/JlTTjklacfHMXr06MJjF6Kr8FWNlyxZEm2Wt3yoAEv2vvI238t4H74yetECobksad7mZTXOwuVFgX1GKEvcK1euLPyu9kCeHiGEEEKUAj30CCGEEKIU6KFHCCGEEKWgIWJ6/EraXAWWU8d97AGntXJlU6+Zso7J+iSn3AKpDplbZV2kcAqiTzWuFT73HIPl47GK4nh8PBanSPqK32WF46OuvfbaaPvxcs0119S0P672y3YOv1p4W+BrwM8dPEfwauxC1As+7pGriHMMjq9+fP7557do1yMnnnhi8prn51NPPbVDv1ueHiGEEEKUAj30CCGEEKIUWGuqB5vZCgBLNtpQtCcDQwh9N96sdagvuwz1Z/dBfdm9aPf+VF92GYV92aqHHiGEEEKIRkXylhBCCCFKgR56hBBCCFEKGu6hx8zWm9lMM5trZrPM7Ctm1nC/o4yY2Y7VvptpZm+Y2Wv0um257KJuMbNdzOyXZrbAzJ42s7vNbO9W7mN7M/vHjjpGUTs0984ys2fM7JCNf0rUG2Uflw0X02Nma0II21TtnQBMBDA9hPBvrt3mIYT3W9qH6HrM7N8BrAkhfIfe69Q+M7MeIYTaFlQTrcIqRbgeBfDTEMKN1fdGAtguhPBw9sPpfgYBuCuEMLwjjlPUjpt7Pwbg6yGEIzfyMVFHaFw2oKeHCSEsB3AugH+yCmeb2SQzmwpgipltbWb/z8yeNLNnzewkADCzYdX3ZprZbDPbq9r299W/YuaY2eld+uNKgpndYmY3mtkTAP7TzEaZ2ePVfrnDzHaotptmZmOqdh8zW1y1N+jL6vt/S+//yMx6VN9fY2bfNbNZAMZ2yY8uB0cDWNc0sQJACGEWgEfM7JrqGHuuaZyZ2TZmNqXqQXiuaawC+DaAPar9WFtVRNEZbAdgNZDtO5jZZWY2z8weMbNfmNm/dNkRC0DjsmsrMrcHIYSF1RtaU3nKjwAYEUJ4y8z+A8DUEMI5ZrY9gCfN7A8AvgjguhDCz6uySg8AxwNYFkL4BACYWa/O/zWlpT+AQ0II681sNoALQggPmtnlAP4NwJcyn92gL81sKIDTARwaQlhnZjcA+ByACQC2BvBECOErHfqLxHAAT7fw/ikARgEYCaAPgKfM7CEAKwCcHEL4o5n1AfC4mU0CcAmA4SGEUZ103KKYLc1sJoCeAPoBOKb6/lq03HdjAJyKSl9vAeAZtHxNiM6j9OOy4R96WuD+EELTOvXjAJxIf130BDAAwGMAvmFm/QH8NoTwkpk9B+C7ZnY1Km67ml19YpP5dfWBpxeA7UMID1bf/ymAX2/ksy315bEARqMycAFgSwDLq+3XA7i93X+BqJXDAPyiKiu+aWYPAjgQwD0A/sPMjgDwAYDdAOzcdYcpWuDPTTc5MxsLYIKZDQdgaLnvDgXwuxDCWgBrzezOLjpusXFKMy4b/qHHzIagciNruqm9x5sBnBpCmOc+9kJVTvkEgLvN7LwQwlQz+wgqHp8rzWxKCOHyjj5+ASDtsyLeR7Mc27PpzRDCRN+XqPT7T0MIl7awn7WK4+kU5gI4rRXtPwegL4DRVe/cYlA/i/oihPBY9S//vqjMmeq7xqD047KhY3rMrC+AGwH839ByRPZ9AC6w6p/7ZnZA9f8hABaGEK4H8DsAI8xsVwB/CiH8DMA1qMhkohMJIbwDYLWZHV5960wATV6fxah4bwAatC31JYApAE6zSqA7zKy3mQ3s+F8giKkAPmxm5za9YWYjALwN4HQz61Edv0cAeBJALwDLqxPr0QCa+utdANt27qGLjWFm+6ISFrAKxX03HcAJZtbTzLYB8MmW9yY6kdKPy0b09DTpylug8tf/rQD+q6DtFQCuBTDbKmnti1AZeJ8GcKaZrQPwBoD/QMWVd42ZfQBgHYD6Xqa2+3IWgBvNbCsACwF8ofr+dwDcVh2sv6f2G/RlNZ7rmwAmV/t9HYD/DZWD7zRCCMHMTgZwrZldjErcx2JU4rO2ATALQADwtRDCG2b2cwB3VmXmGQBerO5nlZlNN7M5AO4JIXy1C36OqNA09wIVb+pZVVm6qO+eqsZ/zAbwJoDnALzTBcctqmhcNmDKuhBCiMbAzLYJIayp/hHzEIBzQwjPdPVxifLSiJ4eIYQQjcF/m9l+qMSB/FQPPKKrkadHCCGEEKWgoQOZhRBCCCFqRQ89QgghhCgFeugRQgghRCnQQ48QQgghSoEeeoQQQghRCvTQI4QQQohS8P8BmzLH8N8ewVcAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 720x720 with 25 Axes>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SuLwEd6qwiMg",
"colab_type": "text"
},
"source": [
"### Build the model\n",
"\n",
"Building the neural network requires configuring the layers of the model, then compiling the model."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VIB9WuvDwoDm",
"colab_type": "text"
},
"source": [
"#### Set up the layers\n",
"\n",
"Most of deep learning consists of chaining together simple layers. Most layers, such as `tf.keras.layers.Dense`, have parameters that are learned during training."
]
},
{
"cell_type": "code",
"metadata": {
"id": "Tt5wpWXWupw9",
"colab_type": "code",
"colab": {}
},
"source": [
"minst_model = keras.Sequential([\n",
" # transforms the format of the images from a two-dimensional array (of 28 by 28 pixels) \n",
" # to a one-dimensional array (of 28 * 28 = 784 pixels). \n",
" keras.layers.Flatten(input_shape=(28, 28)),\n",
"\n",
" # creating two dense network layers\n",
" # first layer consists of 128 nodes, using relu activation for forward-backward propagation\n",
" keras.layers.Dense(128, activation='relu',\n",
" kernel_regularizer=tf.keras.regularizers.l2(0.001)),\n",
"\n",
" # last layer consits of logits array of 10 nodes\n",
" # each node will indicate image belongs to which one among 10 classes. \n",
" keras.layers.Dense(10)\n",
"])"
],
"execution_count": 109,
"outputs": []
},
{
"cell_type": "code",
"metadata": {
"id": "9d5LNXdIuqHk",
"colab_type": "code",
"colab": {}
},
"source": [
"minst_model.compile(optimizer='adam',\n",
" loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
" metrics=['accuracy'])"
],
"execution_count": 110,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "W2RBmcCozMzW",
"colab_type": "text"
},
"source": [
"### Train the model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "NWV4vGkouplK",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 700
},
"outputId": "38140c84-a70e-46c7-e190-2149cb244c09"
},
"source": [
"minst_model.fit(train_images, train_labels, epochs=20)"
],
"execution_count": 111,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.6395 - accuracy: 0.8181\n",
"Epoch 2/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.5075 - accuracy: 0.8495\n",
"Epoch 3/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4792 - accuracy: 0.8566\n",
"Epoch 4/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4612 - accuracy: 0.8607\n",
"Epoch 5/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4501 - accuracy: 0.8638\n",
"Epoch 6/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4392 - accuracy: 0.8675\n",
"Epoch 7/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4341 - accuracy: 0.8700\n",
"Epoch 8/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4266 - accuracy: 0.8697\n",
"Epoch 9/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4210 - accuracy: 0.8719\n",
"Epoch 10/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4180 - accuracy: 0.8737\n",
"Epoch 11/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4142 - accuracy: 0.8737\n",
"Epoch 12/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4100 - accuracy: 0.8759\n",
"Epoch 13/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4110 - accuracy: 0.8746\n",
"Epoch 14/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4077 - accuracy: 0.8763\n",
"Epoch 15/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4042 - accuracy: 0.8778\n",
"Epoch 16/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.4044 - accuracy: 0.8774\n",
"Epoch 17/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.3997 - accuracy: 0.8783\n",
"Epoch 18/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.3984 - accuracy: 0.8778\n",
"Epoch 19/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.3983 - accuracy: 0.8777\n",
"Epoch 20/20\n",
"1875/1875 [==============================] - 3s 2ms/step - loss: 0.3953 - accuracy: 0.8780\n"
],
"name": "stdout"
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tensorflow.python.keras.callbacks.History at 0x7fd2afa5f438>"
]
},
"metadata": {
"tags": []
},
"execution_count": 111
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kUzi6zuYVtfu",
"colab_type": "text"
},
"source": [
"### Evaluate the model on the Test data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "fs_kdfrhupLM",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "e487f639-1e56-4d65-8612-d680192574a2"
},
"source": [
"test_loss, test_acc = minst_model.evaluate(test_images, test_labels, verbose=2)\n",
"\n",
"print('\\nTest accuracy:', test_acc)"
],
"execution_count": 112,
"outputs": [
{
"output_type": "stream",
"text": [
"313/313 - 0s - loss: 0.4441 - accuracy: 0.8642\n",
"\n",
"Test accuracy: 0.8641999959945679\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "j56WsYeZS_4p",
"colab_type": "text"
},
"source": [
"### Use Model for predictions\n",
"\n",
"With the model trained, you can use it to make predictions about some images. The model's linear outputs, [logits](https://developers.google.com/machine-learning/glossary#logits). Attach a softmax layer to convert the logits to probabilities, which are easier to interpret."
]
},
{
"cell_type": "code",
"metadata": {
"id": "6tGUXaW5unbD",
"colab_type": "code",
"colab": {}
},
"source": [
"probability_model = tf.keras.Sequential([minst_model, \n",
" tf.keras.layers.Softmax()])\n",
"\n",
"predictions = probability_model.predict(test_images)"
],
"execution_count": 113,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "eZUYG7pPUMzi",
"colab_type": "text"
},
"source": [
"Here, the model has predicted the label for each image in the testing set.\n",
"\n",
"Each prediction contains an array of 10 numbers. Each number in the array representsthe model's \"confidence\" that the image corresponds to each of the 10 different articles of clothing."
]
},
{
"cell_type": "code",
"metadata": {
"id": "X_rVIutDunLU",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "5f31f3e9-7998-4ac0-c525-94bfee747159"
},
"source": [
"np.argmax(predictions[0])"
],
"execution_count": 114,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"9"
]
},
"metadata": {
"tags": []
},
"execution_count": 114
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ccXm_DqXYPJP",
"colab_type": "text"
},
"source": [
"So, the model is most confident that this image is an ankle boot, or `class_names[9]`. Examining the test label shows that this classification is correct:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "w-O59ojZumwY",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "6753367e-5113-44f5-956a-be6b95b2d807"
},
"source": [
"test_labels[0]"
],
"execution_count": 115,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"9"
]
},
"metadata": {
"tags": []
},
"execution_count": 115
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wGlJu6J4Yx0g",
"colab_type": "text"
},
"source": [
"### Final Representation"
]
},
{
"cell_type": "code",
"metadata": {
"id": "yIQqrB7Tumc7",
"colab_type": "code",
"colab": {}
},
"source": [
"def plot_image(i, predictions_array, true_label, img):\n",
" true_label, img = true_label[i], img[i]\n",
" plt.grid(False)\n",
" plt.xticks([])\n",
" plt.yticks([])\n",
"\n",
" plt.imshow(img, cmap=plt.cm.binary)\n",
"\n",
" predicted_label = np.argmax(predictions_array)\n",
" if predicted_label == true_label:\n",
" color = 'blue'\n",
" else:\n",
" color = 'red'\n",
"\n",
" plt.xlabel(\"{} {:2.0f}% ({})\".format(class_names[predicted_label],\n",
" 100*np.max(predictions_array),\n",
" class_names[true_label]),\n",
" color=color)\n",
"\n",
"def plot_value_array(i, predictions_array, true_label):\n",
" true_label = true_label[i]\n",
" plt.grid(False)\n",
" plt.xticks(range(10))\n",
" plt.yticks([])\n",
" thisplot = plt.bar(range(10), predictions_array, color=\"#777777\")\n",
" plt.ylim([0, 1])\n",
" predicted_label = np.argmax(predictions_array)\n",
"\n",
" thisplot[predicted_label].set_color('red')\n",
" thisplot[true_label].set_color('blue')"
],
"execution_count": 116,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "ohdenMcKZErQ",
"colab_type": "text"
},
"source": [
"Let's plot several images with their predictions. Note that the model can be wrong even when very confident."
]
},
{
"cell_type": "code",
"metadata": {
"id": "_xE1Q9L8Y74m",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 728
},
"outputId": "623788f5-0a4c-48f3-8d66-78bef24c1eb1"
},
"source": [
"num_rows = 5\n",
"num_cols = 3\n",
"\n",
"num_images = num_rows * num_cols\n",
"plt.figure(figsize=(2 * 2 * num_cols, 2 * num_rows))\n",
"\n",
"for i in range(num_images):\n",
" plt.subplot(num_rows, 2 * num_cols, 2 * i + 1)\n",
" plot_image(i, predictions[i], test_labels, test_images)\n",
" plt.subplot(num_rows, 2 * num_cols, 2 * i + 2)\n",
" plot_value_array(i, predictions[i], test_labels)\n",
"\n",
"plt.tight_layout()\n",
"plt.show()"
],
"execution_count": 117,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA1QAAALICAYAAAB4srHRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdebxdVX3///eHzPM8kUDCkMg8F0VkUBQBFdRSgrX4RVulVlvRn1acta1KtVqlrVUrKCqgEkBwQEBBIQpEAgkkJIGQAQJknicysH5/7H1P1vrcc/a5d+cO5977ej4e98H6nLXP3vteOB/2Pnt91rIQggAAAAAArXdAZ58AAAAAAHRV3FABAAAAQEncUAEAAABASdxQAQAAAEBJ3FABAAAAQEm9O/sEgHpGjx4dpkyZ0tmngQKzZ89eG0IY09nnAbQFck7jI+eguymbd+bOlfbsqd3fu7d0/PHlzwv7FOUdbqjQ8KZMmaJHHnmks08DBcxseWefA9BWOiLnjB8vrVpVvW/cOGnlynY9fJdHzkF3UzbvmBX379kjcQnVNoryDkP+AADoYLVupur1AQAaDzdUAAAAAFASN1QAAAAAUFKH1VBR5Nv4KPJFd9OIeefll19O4m3btiXxkCFDSu97+/btSXzAAfu+M+vfv3/p/bYn8g66k0bMOVu2bEniVW5M6cCBA5N49+7dlXa/fv2SPp+/9u7dW/O4u3btSuLDDjus/sl2AHIO2kOH3VAxsUDjo8gX3U0j5h1/cTNr1qwkPuecc0rv+9FHH03iwYMHV9rTpk0rvd/2RN5Bd9JZOSeEkMQWzVTwu9/9Lum75pprkviEE05I4pXRjCiHH3540rd169Yk3rBhQxL37r3vsnLp0qVJ32233Vb13DsaOQftgSF/AAAAAFASN1QAAAAAUBLrUAHAftq5c2cSf+Mb30jim266qdL2Q2TWrFmTxAMGDEhiv30RXycVx/FQHEk688wzk/i9731vEp933nktPi6AzlU05O9zn/tc0vfHP/4xie+4446a+x06dGgS+zrNPW5F2Th/7dixI+n75S9/mcRvfvObax4X6Gp4QgUAAAAAJXFDBQAAAAAlMeQPAFrp4x//eBJ/97vfTeLNmzcncTwtsR/SN2LEiCT2w2QGDRpUafspiv2Uxn7f8TCgl156Ken71a9+lcR+2M9pp51Wad9///0C0LjiJRK8uXPnJrHPOWPGpDOIx0s5+JwzcuTIJO7Tp08Sxzln8eLFSd/ChQuTmCF/6E54QgUAAAAAJXFDBQAAAAAlcUMFAAAAACVRQwUALRDXSX3lK19J+saPH5/Ecd2TlE5h7Kc33r17dxIXTX0e70dqXjfhpzAu2u/gwYOTuFevXkkcT638lre8Jen7xS9+UfM4ABrL1q1bk3j06NFJ7Gs+X3755Urb12nGfdX27bePPffcc/VPFuiieEIFAAAAACVxQwUAAAAAJXFDBQAAAAAlUUMFAC3wmc98ptIeOnRo0udrm/zaLStXrqy53+HDhyexr3Xq3Xtfmvb1Cjt37kziUaNG1TyPeD9S83WpfG3XuHHjKm2/DtXatWuT2NdkAOhcq1atqtnnc4HPXzFfl+nXnfK1l/G+fJ5cvXp1zeMAXR1PqAAAAACgJG6oAAAAAKAkbqgAAAAAoCRqqACgBTZt2lRp+7VWfP2Rr5l6//vfX2lfccUVSd9JJ52UxH4NqxUrVlTaQ4YMSfomT56cxL5uIj7PeD+SNHHixJrbStKWLVsq7R07diR9S5YsSWJqqIDGMm/evJp9ffv2TWL/+Y7rony9lV+Hyue+ojWsfO0l0J3whAoAAAAASuKGCkBi/HjJrPbP+PGdfYYAAACNgyF/ABIFs+22qL+7iqcZ91Ob+2Ev3pe//OVKe9iwYUmfH0Kzffv2JD777LMr7fvuu6/wOEceeWQSL1y4sNLevHlz0vfNb34zieNp4SVpzJgxlbafBn7mzJlJfOqppxaeF4CONXfu3ErbD/Hz+cvnnHg5hnios9R8aQY/5XqcC/3SDH44M9Cd8IQKAAAAAErihgoAAAAASuKGCgAAAABKooaqAfj6hAMOSO9z/RjlmB+j7KcpffrppyvtqVOnlj1FoMfZtWtXzT7/mfSfQ+9d73pXpX377bcXbrthw4YkjuumPvvZzyZ9Q4cOTeKf/OQnSbx+/fpKe/ny5Unf9OnTk9jXUMV5yU+dPGfOnKrnDqAx/PnPf660/TWFr5nyn++4bsov6+A/+yNGjEji+BrEH+eggw6qd9pAl8UTKgAAAAAoiRsqAAAAACiJGyoAAAAAKIkaqlaI11fw6874McrPP/98Ej/44IOV9vnnn5/07c/aDL5myrv11lsr7Y9//OOljwP0NC+88ELNPv9537FjR+G+VqxY0eLj3nzzzTX7LrvssiQeMGBAEvt6zOOPP77SfvHFF5O+wYMHt/icvLg2E0DjWbBgQaXdp0+fpM/nr61btybxhAkTKu2HHnoo6fP1o34dvTjes2dP0jdy5Mh6pw10WTyhAgAAAICSuKECAAAAgJK4oQIAAACAkqihKsmPQfYeeOCBJH744YcrbV+b8U//9E+lz2P16tVJfNdddyXxkCFDSu8b6MnWrFnT4m19rYCvWYg/877mwDvrrLNq9r3xjW9M4qVLlyaxr1G48847K+2zzz476Yvrq6TmNVXxefbq1SvpW7lyZc1zBND54rWk/Oe3Xg3V29/+9hYfx+e+gQMH1ty2aG0/oKvjCRUAAAAAlMQNFQAAAACUxJC/VoinJO7dO/3T/fnPf07ieMpSSRo3blyl7accftvb3pbEI0aMSOKdO3dW2pMnT0761q1bl8SbN29O4okTJwpA6/mlD2J+2QTPD3uJh8j54TZ+X4sWLUrieLmDJUuWFB73yCOPTOKFCxdW2s8++2zS961vfSuJ/fTIcR7yyzMU/W0AdL5Vq1ZV2q1dmuUd73hHzT6fC9avX5/Eo0ePrvne7du3t+o8gK6EJ1QAAAAAUBI3VAAAAABQEjdUAAAAAFASNVQF/PTGcd3Utm3bkr4ZM2YksR9nHNdBbdmyJenzNRRF8fz585O+SZMmJbGvv4rrvgC0XNG06X4aYj91sI/jKck/+clPFm579913J/HcuXMrbf/59zWTcc2UlNZfTZ8+PembM2eOisT5z8ySvt27dxe+F0Dn2rFjR6Xtl0+pd13w2te+tmbfaaedlsQPPvhgEvt8Fhs1alThcYGujCdUAAAAAFASN1QAAAAAUBI3VAAAAABQUpevofL1Rn6sv6+Divv9tn5csa+TiH37299O4nidKUnq379/Ei9fvrzSjuupqr3Xj0GOz9OvJ+FrtTZt2pTEL730UqXt675auzYF0JO8+OKLNfvqrSXlP8PDhg2rtL/85S8XHjfeVkrzw5NPPln43vHjxyfx2rVrK22fk+opWnevaFupOHcC6Fy+BtJ/vv11RWzKlClJPHPmzCQuWqPP5zagO+EJFQAAAACUxA0VAAAAAJTEDRUAAAAAlNQlaqiK6qR8HZTnax1irR33f9NNN1XaK1euTPpOPPHEJPY1FBs3bqy0R44cmfT5tRniugdJ2rp1a839ev5vtX379kr76aefTvpOOOGEwn0BPVnROlRe3759k/h1r3tdEj/wwAOVtl87zueduO5RSvNUvJ5VNT4/xPVXfr9+X8OHD0/ieJ0qn7O8ZcuWJfFhhx1WuD2AjuOvk3bt2pXErfm8+vzlr6PqXZMB3RVPqAAAAACgJG6oAAAAAKCkLjHkr+gRsp8W3cd+OE28r3pD/K677rokfuqppyrtgw46KOlbt25dEvuhdzt27Ki0J06cmPRt2bKl5jlK0sCBAyttP+V6vWnjY3fddVcSM+QPqC0epuv5z6z/TF9++eVJfOedd1ba8ee5mno5rYj//MdDAP2QPz9V8tvf/vYkjof81eOHKTPkD2gc/rPul1A5+uijW7yvCy64IIm/8pWvJHFr8hXQnfCECgAAAABK4oYKAAAAAErihgoAAAAASmqIGqp6Y259XUBcN+SnRS+aJt174YUXkvjWW29N4rjuSZKmTp1aacdTmUvN6xN8TVWfPn0qbf/7xFObVxP/Tv369avZJ0mDBg1K4vhYf/zjHwuPA2Af/xmO+dwwduzYJB4xYkTN98a5QGo+1bnPD63Jaf698ZTGvs/nrFe+8pU19+vPoX///klM3QTQuPzU5r72+tBDD23xvo4//vgk9lOwFy3t4q9PgO6EJ1QAAAAAUBI3VAAAAABQEjdUAAAAAFBSh9ZQxeN44zWgWlMjIBWvtbRmzZokXrZsWRIvWrSo0n7xxReTvr59+ybx0KFDkzhel2bz5s1J3+7du5PY1yfEv68/Jz/mePjw4TXPq95Y6AEDBiRxvP3gwYOTvnnz5glAdX4dqrhuyK8H52sDFixYUHO/fk0Ynzu8onznFa1L5/fjf7/WrPfnj+PXoQLQuSZNmlRp+3Wn/DXXgQce2OL9+vzlUUOFnoonVAAAAABQEjdUAAAAAFASN1QAAAAAUFKH1lDFdUSxVatWJfHy5cuT2I//jWO/HszSpUuT2K/xFI//HTJkSNLn6wQ2bdqUxPGx/DhifxxfyxSvH+XXbZgwYUIS+/qseN9+fRu/Htb69euTOK6bWrlyZeG2APZpzdpKr3jFK5L4mWeeqbmtr1Xyxylad68e/964nsGvYef369fSKjpH/15fuwqgc8Wf5yVLliR9vs7pqaeeavF+fa25V1RjVW/NTaAr4wkVAAAAAJTEDRUAAAAAlNShQ/5iv/3tbyvtF154Ienzj4z9cJJa069Xe68f1hcPkfND4PwwFj/1eTzczg+B8UPv/PTm8XShfvpyP016a4bP+CGAfjrUeJiiH2pYb/pToCfz05kXfV78kL8//OEPNbctmlZYap6H4lxTb4kJ/944rjXkukk8zbKP602L7vMfgM516qmnVtp+GQc//HfOnDltdlx/3VR0XKA74QkVAAAAAJTEDRUAAAAAlMQNFQAAAACU1GFFNJs3b9bdd99dia+99tpK+4gjjki29dOIF01v7qfw9LVLvqYg3pevKfL1CVu2bKm5Lz9du5+u2J9HXK/lp4l/8sknk9ifl99XzNdj+Snm+/fvX3PbommSgZ7OL31QVIPkc8fChQuTuE+fPpV20ee5tfy+fB6K43o1k4sXL07i8ePHV9q+3jT+fSSmQwYazZlnnllpf//730/6/HXTY489Vvo4PvcV1YjWqwEFujL+6wYAAACAkrihAgAAAICSuKECAAAAgJI6rIZq0KBByboIDz30UKX9xBNPJNvOnDmzcF/x+H1fXzVy5MjCeNiwYZW2r1Xy9Vbr1q1L4kWLFlXavmZg8+bNSexrGebOnVtpH3fccUnflClTkviee+5J4nhdh3pjkH2dxIEHHlhpDx06NOnzNWIA9vGfpaLaJ79m1fr165N44MCBlbZfw641fF6pJ677qrf+1e23357EcV569NFHkz6fhzZs2NCq8wLQvl796ldX2nEttdS8HnR/6qn9dYW/jortT+4DGh1PqAAAAACgJG6oAAAAAKAkbqgAAAAAoKQOq6Hq1auXhg8fXok/+9nP1tx269atSfzwww8ncVzL9Kc//SnpW7ZsWRI//vjjSRyv0+TH+vr6BF8nENdjHXvssUnf61//+iS+4IILktiPYS5y4YUXJvGzzz5baY8aNSrp8+OXfU1ZXAfSr1+/pG/atGktPiegp/Gf/507d9bc1q87Fdc9Sulnz9db+XqGohoE31cvh8Xq1S/43BnXes6YMaPwOP53AtC5Jk+eXGn76wSfn3xuW7JkSaV96KGHFh7Hr0lXlAvacg0+oNHwhAoAAAAASuKGCgAAAABK6rAhf60xePDgJD7nnHNqxv/wD//QIefUke64447OPgWgx/NDZIuGzPlpw/0Qmnhffoif54caxrEfalcvjocE+uGB8RISkvTggw8mcdGQYH+cHTt21NwWQOfyQ/z8Egp+CZnWDPmbMGFCEsdDh0eMGJH0MeQP3RlPqAAAAACgJG6oAAAAAKAkbqgAAAAAoKSGrKECgM7mpwMeOHBgpe2XdvjIRz6SxL/97W+TOK4x8jVS9cT1SkU1UtXEdV/+uJs2bUris88+O4nf/OY3V9pf+MIXkj5fB+ZrNAB0rKIlFN72trclfTfeeGMS+/rQmTNnVtp+SRgvzov1zsnXVAHdCU+oAAAAAKAkbqgAAAAAoCRuqAAAAACgJGqoAKCKbdu2JXFcN+Trq3bv3p3EY8aMSeKnn3660vbruhStb9VaRXUU/pz92lljx45N4tGjR9c8jq/HWr58eavOE0DbKvrsX3TRRUnf9ddfn8R9+/ZN4ltuuaXS/vznP194XL+2VFHNp1/bD+hOeEIFAAAAACVxQwUAAAAAJXFDBQAAAAAlUUMFAFWcfvrpSfzggw9W2v3790/6pk2blsRPPfVU+51YB1myZEmlPWTIkKTPrzt16qmndsg5AajO12LGdY7nn39+0ufXg/Kf59aslXfMMcck8RNPPFFp+zz54osvtni/QFfDEyoAAAAAKIknVEA3cNVVVxX2X3311R10JgAAAD0LN1RAJ+AGqPH5YWw7duyotP00w60ZItNVxFPB+yFBu3btSuJBgwZ1yDkBqC5e1qGeyZMnJ/FDDz2UxNu3b6+0//SnPyV9r371q5PYT5u+c+fOStvnibVr17b4HIGupvtdBQAAAABAB+GGCgAAAABK4oYKAAAAAEqihgoAqpg4cWISn3jiiZW2nw64Xg3Rnj17Km1f6xBCKHuK+8Uf15/X4YcfXmm/6U1vSvo2btyYxKeddlobnx2A1jCzFm/73ve+N4mPOOKIJL700ksrbV8z5V122WVJvGnTpkp78ODBSd8ZZ5zR4nMEuhqeUAEAAABASdxQAQAAAEBJ3FABAAAAQEkdVkM1e/bstWa2vKOOh1Im198E6Do6Ku/4WoGe5r777tuft5N30G10h2udH/zgB+2y3xtuuKFd9lsCOQdtrsNuqEIIYzrqWAAgkXcAdCxyDtAzMeQPAAAAAErihgoAAAAASrLOWgMFaCkzWyOpaEz6aElrW7Crztiup5zbZIa6oLtoQc6Ruv5ntpGP2ZLtyDnoVtow7zTqZ7azjtnS7fbvWieEUPdHCm+VQpDCES3cfpkURld5fWtL3l92+4L9XC6FA2v0/VQKc/KfZVKYk7/+zuj1OVJ4WQonSKGfFH4jhXlS+IdoP9+Vwkl1/oafda/NkcJPWvE7/HeV1z8vhY+24m/Rqu0L9jNFCn8dxcdK4Qdt8e+r9eeiRxp1u550bp39I4W9+WdqnhRulsLAOtv/Xgqn5O2qOasdz/V1Ung0P9frpdA7f32EFG6TwuNSmCWFY/LXx0hhZr79W6P93F4rt+X9V0rhXXn7B1JYmv+NHpXCaXXOsZJz2ipv1DneB6Xwns7+76jl59v1P7ONeszWbNe5/w2EUdE1wkopPB/FfRvg/P5KCvPz65dTXN8npLBYCouk8Mbo9fPy1xZL4aro9RvyvPSl6LVPx/moyvFPlMK1Unh39HfZJYUn8vbVnfi3GSOF33T2v6PWn3fX/sx2h3Or9dPSIX/vkDQz/2dXdLmkA6t1hKDpIeiEEHSCpFsk3Zq/fkP0+mWSloagOZLeqOxvcVz+usx0vKReIejRgnP4Z0nfagrMdKSkXpLOMNOg/fz9OsMUSX/dFISgJyRNMtPBnXZG6Ol25J/ZYyTtkvT3nX1CkmQmM9uXa/P29ZIuzc91uaT/l3d/UtKcEHScpHdJ+mb++jskfVvSqZKuzPfzFkmPhaAXahy3t6T3SLoxevljeU67StJ32uY33H/5uV4n6R87+1yAlgpB66LrhG9L+s+mOATtyv+77jBm6uVemifp7ZLud9sdJelSSUdLOk/St8zUK3///0g6X9JRkt5hpqPMdJyy/HqcpL8w0zAzTZD0yhD084JT+qSka0LQ96O/0wuSXpvHVxWce7sxU+8QtEbSi2Y6vaOOi+6t7g2VmQZLeo2kv1X2AWx6/Wwz/d5MM8y00Ew3mMnceweY6U4zvbfKfj9mpj+b6XEzfaHg+P9ppvlm+p2ZxuSvnWCmh/L33mamEbVeN9PFkk6RdIOZ5phpQI3jmKRLJN1Upfsdkn6St3dLGiipj1T5ff9V0mcKfodpkl4KIXmU+A5JP5J0t6SLom1/b6Z/N9MsMz1lpjOq7O9NZnrQTKPd64eZ6Tdmmm2mB8x0RI1TOj5//9NN/27yi76vmmmemZ4w0/Si1yVdrexmcI6ZPpy/9gtF/40AnegBSYfneeqXTS+a6b/NdHnRG830kfy/93lmlZuXq830gWibz5vpo3m7WS4z0xQzLTLTD5Vd1BwUHWKUpF0h6Kk8vkfSX+btoyTdK0khaKGkKWYap315p5+kvfmF2pWSvlLwq7xO0qMhaE+VvvslHZ6f67KmXGKmU8z0+zp/n2p59ggzzYq2mWKmJ/L2yWb6Q56X7sovxJpy3TfM9IikD4Wg7ZKWmenUouMDjcxMPzDTt830sKSvFFyv/N5Mp+Tt0WZalrePzv//Pyd/z9T89b+JXv+O5TcgZtpqpq+Zaa6k0+JzCUELQtCiKqd5kaSfhKCXQtBSSYuVfVlzqqTFIWhJCNql7LrnImX5Z4BlXwb1kbRX0r9I+lzB32GIpONC0NyCbZJzr5F7p5hpXvSej5rp83n7n8z0ZP53+kn+2iAzXZf/rR4zy66vzHS5me4w072Sfpfv7ueS3lnr/IDWaMkTqosk/Sb/n/86M50c9Z2o7H/qR0k6VEru9Acru8C+KQT9X7xDM50raaqyD+8Jkk4205lVjj1I0iMh6GhJf9C+D+8PJX08/7bkiaLXQ9AMSY9Iemf+jciOGr/nGZJWhaCnq/RN174brXuUPZ15SNI1ZrpQ2UVL1W+Jc6dLzZ5eTVeWrG5S8yd/vUOofBOdJCwzvU3Zt8sXuBs0SfqupH8MQSdL+qiiJ2LOccoutk6T9FkzHajsW6wTJB0v6fWSvppf+NR6/SpJD+R/0//M9/uI1PwGsAN8t4G360nn1hDym43zleWA1r73ZEnvlvRKSa+S9F4znSjpp8q+cGlyiaSf1sllUyV9KwQdHUIyLn6tpN5NF1OSLta+G665yj5zym8sJkuapOwp00XK8s+XJP2DpB/lNyG1nC5pdo2+t6jE3ydXLc8ulNTXTIfk20xX9vfpI+m/JF2c56XrJH0x2lffEHRKCPpaHndWDimjO3xmG/WYrdmuEU2S9OoQ9BHVvl6p5e8lfTN/mnOKpBWWjWiZLun0/PW92ncjMEjSwyHo+BA0s4XnN1HSc1G8In+t6ushaIGkNcquY36h7MuYA+qMyjlF2ncjVEPl3CXtUPXcW+QqSSfmf9umEQmfknRvfg31WmXXLE2jgE5SlovOyuOulG+adPXPbHc4t6pa8jj6Hdo37OQnedz0P+lZIWiFJJlpjrIbjaYP9O2SvhKCqq3kdm7+81geD1Z28XG/2+5lZRcykvRjSbeaaZik4SHoD/nr10u6udbrLfj94t+z2dMpM71S0vYQssSQf9v713lfH0l3SbrITF+XdLCkH4agO9xuJihLRk37PEXS2hD0rJmel3SdmUaGoPX5Jrfm/5yt7G/a5HXKktS5IWizO8/Bkl6t7G/RpF+N3/X2/MZyh5nuU3Yx+BplN797Ja0y0x8k/UXB65ur7He1agytbE8hhBZ9CDpju550bg1gQJ6HpOwJ1bXKPhOt8RpJt4WgbZJkplslnRGCrjHT2PzLhzGSNoSg58z0IVXPZc9KWh6CHvIHCEHBTJdK+k8z9VP2lHpv3n21pG/mv8cT+X73hqBNkt6Un9MIZRcSbzPT/0kaIelrIehBd6gJkha4175qpk8ry0d/28q/jerk2Z8pu+i7Ov/ndEmvkHSMpHvyvNRL0ovRLn+q1Gqp5pP1htIdPrONeszWbNegbg5Be0telzwo6VNmmiTp1hD0tJnOkXSypD/nn6MByj4rUpY7bmnrX8ALIXtiJElm+oWkK8z0KWVftt7jvziXu+6pIT73qrlXanY9FXtc2eijn0uVoYfnSrrQ8hEEkvpLlVKEe6LrLKmTrln2R1f/zHaHc6ul8IbKTCOVXcQfa6ag7H+GwUwfyzd5Kdp8r9vfHyWdZ6YbQ5CfStAkfTmEVo/hb5cpCfNvtN8uJU/fmlyq6sMApexb4h8q+zZlk7ILiHvVPAHskDQsit8h6YimR/yShiob8tOUkJr+rv5v+oyyJ4HTlH2zEjtA0sb826t6/N+xrf6u/aWaTwCB9rbD//dvpj1Kn8T334/936zsadJ47bsRqJrLzDRFyi4Mqslvfs7Itz1X2Wda+Rcl785fN0lLJS1xb/+Msqc8TbWtM5R9CfNGt90ONf99P5Y/tY/Ff6P9+fv8VNkXOrdKCvmF4LGS5oeQDkWK+L8ROQTdQc3PfqTq5y4E3ZgPF3yTpF+b6Qpleeb6EPSJKvvZmX/h2RrPKx2GPCl/TQWvS5LyIXSzlX15dFgIusSyobw3uCfm1fJPmXMvyuFvknSmsifun8rzjUn6Sz/UMf9ynHyDdlNvyN/FyoaVTA5BU0LQQcr+B9+SR6SflbRBWYGjd5ek9+RPVWSmiWYaW+P8Ls7bfy1pZv5N7QbbV1t0maQ/1Ho9b2+RNKTgXF8vaWHT07Ym+XjhS7SvfiruGyHpzcpuqAYqe5oWpKo1Wgu0r16haZ/H5n/TKcqG8rRkwo/lym68fmimo+OO/EJsqZn+Kj+OWTZZRjUXmam/mUZJOlvSn5V9oz/dssLUMcqS1KyC16v9Taep/iN+oCMtl3SUmfqZabikc+ps/4Ckt5ppYD5M5G35a1J2w3CpspzU9C1zS3NZommb/AnVx5UVtMtMw83UN9/s7yTdHz+NtqyeYlII+r1akXfqWKZ9Xyb9ZcF2KsqzIegZZV8CfUb7bjgXSRpjlt1QmamPz10OOQTdRp3rkmXa97lrus6RmQ6VtCQEXaNspM9xymp+Lo7yxkgzTd6PU7tD0qV5XjxE2VP1WcquBaaa6ZA8D12q6AvifFROU1Y/3OAAACAASURBVO3mAO37MraXVMlbTVqaf5rUyr2rJI0106g8X745P5cDJB0Ugu5TlkOHKbvJu0vSP+ZfSKnOsEHyDdpMvRuqd0i6zb12i1o+29+HlA3DSQqnQ9DdymoCHrSscHmGqt/wbJN0qmUFia9TVgQpZTNifdVMjyurW6j3+g8kfdtqT0pR6ynUmZKeC6HZN8RSdsP4xRD0srIP8BnKhuj8qMq290s6Mf+AnyHpeVdzdb+yi74JVd6byGsV3qnsm+DDXPc7Jf2tZQWe8xVNduE8Luk+ZXVg/5qfy23563OVPWX75xC0suD1x5UVx8+1fZNSvFbSr+r9Dm3JzM4zs0VmttjMrqqxzXVmttrMChOnmR1kZveZ2ZNmNt/MPlRlm/5mNsvM5ubb1JxQJd++l5k9Zma/LNhmmZk9YWZzzMw/eWzaZriZzTCzhWa2wMyafeNvZq/I99H0s9nMrqyxvw/n5z/PzG4ys6rfJJrZh/Jt5tfaVyMLQc8pG4o2L//nY3W2f1RZvpgl6WFJ3wshe08Imq8sTz0fQjZsrRW5zPuYmRYo+xz9IoRsIgpJR0qaZ6ZFyurA/H+DX1RWIyBlOev9yi6Cvqnm7pSq1qZ6X1A2zPARqUXfdNfKs1J2I/U3yv7WygvbL5b073lemqPiYZinK6sTa2htlXdaknPy7Vqcd9oq5+TbtVne6Sk5p4pan5f/kPR+Mz0mJRNMXaIsB8xRNlz2hyHoSUmflnR3vp97pPrXC2Z6m5lWKKuX/pWZ7pIquexnkp6U9BtJHwhBe/OShg8qu6ZZIOln+bZNPqDsSdl2ZblrYJ73ZoegjfGx82uVYWYtyoc1c28I2q3sbzYr/70X5m/pJenH+fEfUzab4EZlk4T1kfS4mebncS0dfs1SVk+51sm3K8w7DXutsz9zrvPT8h8pfFMKr+/s82jH36+fFB5Svp5OxxxTvbRvGGRfZTd9R1XZ7kxlxajz6uxvgqST8vYQSU/5/SkbTjA4b/dRlvhfVbDPjyi74P5lwTbLJBWugaRs7P3f5e2+koa34G+zUtkidL5vorInzQPy+GeSLq+y3THKbkQGKht6+ltJh3f2f2v8tPxH2ZpWUzv7PFp4ridK4UedfR71z7Pt8k5Lck7e1+K801Y5J9+uTfIOOadn/kjhw1L4u84+j4Lzu18KIzr7POqfZ8+51sm3a3HeaaRrnZauQ4X99yVl/7K6q4MlXRWqT9HcXvIpXsOSEEI8xWsihHC/lBSiVhVCeDGE8Gje3qLsG7qJbpsQQtiah33yn6o1aGY2SdkY7++1+Deqvp9hyhLltfk57AohbCx+l86R9EwIodaq670lDTCz3sr+u6w2S+WRkh4OIWwPIexRNlTl7WV+B3Saq9SCb7IbxGgVLD/RQNos77Qk5+R9Lco7bZVz8n21dd4h5/Q8/6u01r5h5CUMXw9BGzr7XFqgR1zr5Ptqbd5pmGsdbqg6SAhaFZrP/tdthKCnQyhev6Yd1Jr6db+Z2RRlywI8XKWvl5nNUTZD0D0hhGbb5L6hbEHnl+scLki628xmm9n7qvQfomy2pO/nj9S/Z2b1FoOuOZlKCOF5ZcNNnlU249qmEMLdVTadJ+kMMxtlZgMlXaC0YBkNLgQtCqHZ7KkNKQTdE0Jlop5G1i55pyjn5P0tyTttlXOkNsw75JyeKQTtDKFqGUSnC0FrQvGixI2kp1zrSK3POw1zrcMNFeCY2WBltYJXhhCaTQ8fQtgbQjhB2QxIp5rZMVX28WZJq0MItdYBir0mhHCSspqZD5iZr3vprewx/v+GEE5UVltYdQx1fuy+ki5Ujel5zWyEsm+3DlE2ZewgM/sbv10IYYGkf1c2rfdvlNW/tHY2KQB11Ms5Uv2808Y5R2rDvEPOARpPA17rSK3IO412rcMNFbqyoqlfSzGzPsoSzA0hhFuLts0fQ98n6bwq3adLutDMlil7PP86M/txjf08n/9ztbJJQE51m6yQtCL6dmiGsoRTy/mSHg0hrKrR/3pJS0MIa0IIu5VNuV11ooAQwrUhhJNDCGcqm7XzqYLjAj1Bm+ad1uQcqTDvtGXOkdo275BzgPJ6yrWO1Lq801DXOtxQoSvLp3i1Q/JvKpIpXlvLzEzZuN0FIYSv19hmjJkNz9sDJL1B+2YdqgghfCKEMCmEMCU/r3tDCM2+GTGzQWY2pKmtbFHCeW5fKyU9Z2avyF86R9nsTLVUXaQ68qykV5nZwPx3PkfNF4BtOr98ml47WNmY4hsL9gv0BG2Wd1qSc/Lt6uadtsw5+f7aMu+Qc4DyesS1Tr6/1uSdhrrWKVzYF2hkIYQ9ZtY0xWsvSdeFEOb77czsJmXrbY02sxWSPhdCuLbKLk9Xtk7IE/m4YUn6ZAjh19E2EyRdb2a9lH0h8bMQQs1pQltgnKTbss+6eku6MYTwmyrb/aOkG/JkukT54q9enqjeIOmKWgcMITxsZjMkPaps0cTHJNVaIfwWMxslabekD7SgKB3o1to477Qk50htm3damnOkNso75BygvB52rSO1IO804rWOhVB10g4AAAAAQB0M+QMAAACAkrihAgAAAICSuKECAAAAgJK4oQIAAACAkrihAgAAAICSuKECAAAAgJK4oQIAAACAkrihAgAAAICSuKECAAAAgJK4oQIAAACAkrihAgAAAICSenf2CQD1jB49OkyZMqXV75s7V9qzp3Z/797S8ceXPy/sM3v27LUhhDGdfR5AWyibc9BxyDnobsg7ja8o73BDhYY3ZcoUPfLII61+n1lx/549UondogozW97Z5wC0lbI5Bx2HnIPupmzeGT9eWrWqdv+4cdLKlftxYqgoyjsM+QMAAAC6oKKbqZb0o21wQwUAAAAAJXFDBQAAAAAldVgNVaMU2+2JZilYs2ZN0terV68kPuCA2vebftt6QgiVdu/e6Z99yJAhSWz1in/aCUW+6G4aJe8U2bZtWxK//PLLhXERv22fPn0q7cGDB5c4u/ZH3kF30og5Z9GiRUnsrzF8HF+v9O3bt2afJO3evTuJi66b/HunTp1ac9v2RM5Be+iwG6pGKfKNb6K+853vJH3Dhw9P4gEDBtTcz7Bhw5LYJ6S9e/cm8a5duyrtsWPHJn1nn312EvsE1lEo8kV305q8429G/IWBvxiI7c+XIA8++GASb9++PYnj3OHzivfSSy8l8Zgx+64ZzjzzzLKn2K7IO+hOGuVaJ+avMfwXwv369UvinTt3Vtr+5jDuk6RVrkAn/oLY5ysf//rXv6590u2InIP2wJA/AAAAACiJGyoAAAAAKKnHrUN18803V9r/9m//lvSNGDEiiSdMmJDES5curbQnTpyY9E2bNi2JFyxYkMT9+/evtF//+tcnff6R+WWXXVb13AG0n6I6gnrbelu2bEnie++9t9J+9NFHk74777wziV/xilfUPNbWrVuTvnXr1iXxqFGjkjgenvPFL34x6XvLW96SxBdeeGESH3zwwQLQNW3evLnSnj9/ftIXDwWuZseOHZX2M888k/TF1zJS86HRAwcOrLTj4cotOS7QlfGECgAAAABK4oYKAAAAAErqcUP+4ln+/Ow1RdN9StL48eMrbT9bjR96s2nTpiQeOnRopf38888nfUcccUThcQG0v3pD/oqG+X33u99NYj9NcTyDoP+8T58+PYnnzJmTxPEMXPGyD1Lz4YF+CYZBgwZV2n6ZiOXL04muPvzhD9d879VXX530HXjggQLQuOLhvvVmIfYzC8exL4Xw742HFkrpdZS/xiqaORno6nhCBQAAAAAlcUMFAAAAACVxQwUAAAAAJfW4Gqq41slP4emnBx05cmQSx1Mh+1qFjRs3JrGvv4jHGftarWOPPbbeaQNoZ62pmfrWt76VxOvXr0/iQw45JIn79OlTafsahLFjxybxWWedlcS33nprpR3XcUrNax+Kcoufnn3q1KlJPGzYsCSOa6w+/elPJ33XXXedADSuW265pdL2Nd6TJk1KYp+T4prPuIbT90npFOtSWufpa8lfeOGFJJ49e3YSn3zyyQK6Kp5QAQAAAEBJ3FABAAAAQEncUAEAAABAST2uhmry5MmV9ty5c5O+Xr16Fcbxuiy+dsGPQfa1Dhs2bKi0/Rhk1qECOl+9GqrnnnuualuSDj300CTeunVrzePEeUSSVq1alcSHHXZYzfjpp59O+nyd5ytf+cokvv/++yttv3ZUvE6NJG3fvj2J4zVjVq5cmfT96Ec/SuLLLrssieO/ZVEtGoD28b3vfa/SnjBhQtLn6zZ9Durde9+loc91AwcOTGJ/ndS/f/+q+5Gk1atXJ/GsWbOSmBoqdGU8oQIAAACAkrihAgAAAICSuKECAAAAgJJ6XA1VPJ7fr//kaxt8TUW8TlVcEyU1r4uaNm1azXPwNRJ+nDGAjufXcPIWL15cafu6gXjtFUkaPHhwEr/00kuVtq+39Nv6Ne3OP//8SnvmzJlJX1znVO084tjXdW7bti2J43X2JGnXrl2Vtl+L5rHHHktiX0NF3RTQuRYtWlRpn3LKKUmfXztq9+7dSRxfz/j8FOcFqXnOidez82vb+Rzr16UCujKeUAEAAABASdxQAQAAAEBJPW6sWfzI+aCDDkr6jjrqqCT2w1ZuvvnmSnv9+vVJ3/z585P4zDPPTOJ4OtCJEycmff4Rup+WFEDniz/j8dTAUjqkT2o+XDj+TPvhwX744ObNm5M4nvL43HPPLXyvjw8//PCa5+SnQvdDd/y06jE/3TGAzvXiiy8mcTy02E+T7qcv90Px4mVh/LTpPvf5IYHx8EGfU/x7/VBioCvjCRUAAAAAlMQNFQAAAACUxA0VAAAAAJTU42qojjzyyEr7d7/7Xc0+qfn43qOPPrrSPvXUU5O+973vfUl88MEHJ/GkSZMq7REjRiR9fupjAI1nxYoVlfbQoUOTPl9D5Y0bN67S3r59e9Ln6wz69OmTxHHtll/qwS/fcOCBByZxPC2xn4591apVSeynVY+Pe8ghhyR9o0aNSmJfBxrXYABof74msqgW29dT+muQtWvXVtp+yvV58+Yl8datW5M4rqnyS0T4Gk9fUwV0ZTyhAgAAAICSuKECAAAAgJK4oQIAAACAknpcDVVcvzBo0KCkz49B9rVOMV/34Gso/Foz8Vjh3r3TP7tf74W1GYDO52uMYr5uwNcnHXfccUkc10X5ugLP1xnE+cAfx9cu+dqIeE0Yv06NzzN+X/5YMZ/fHn/88ST2dRcA2tdTTz2VxHHO8dc6nl9zM84VzzzzTNJ34oknJvGiRYuSePLkyZW2r6X01z5c66A74QkVAAAAAJTEDRUAAAAAlMQNFQAAAACU1ONqqOKxxH49mAMOSO8v4zVcpLRu6oQTTkj6/BjkHTt2JHFcn+BrKPy6MwA635IlS5I4Xl/F1z1u27YtiX0+WL9+faUd1zVV25cX1yv5+ip/nNWrV9fs98fx5+HzYfz7+hpRXwuxdOnSJKaGCuhYCxcuTOJ4HSqfn3we8fWSY8aMqXmcV73qVUk8Z86cJI5zjs8bPl+xXh26E55QAQAAAEBJ3FABAAAAQEk9bsjfgAEDKm0/xC8e4lJN3O+nDvX88Jn4uH6qUIb8AY3nueeeS+J46QM/bbi3fPnyJJ4yZUql7Ye5+CHAfkmGIUOGVNo+V/jj+POKh+bF51/tuH7ZiHh4tD+uj/3UyQA61uLFi5N42LBhlbZfEsF/fn15w+WXX17zOO95z3uS+Nvf/nYSF+VGP9TQx0BXxhMqAAAAACiJGyoAAAAAKIkbKgAAAAAoqcfVUMVjdv04Yj+lp4+LaqziGimp+ZTEcT0C44iBxufrCuKay6FDhyZ9fnrgLVu21Hyvr5Hyn3/fH7/XH8fXK8T1VpK0YcOGStvXUPmlHfzvtGbNmko7rseodty5c+cKQOfZvHlzEsfXJP5axl+f+PjKK6+seZy/+Iu/SGK/76JlHnz9ONc+6E54QgUAAAAAJXFDBQAAAAAlcUMFAAAAACX1uBqq0aNHV9pFY3+l5ms3+BqEmK9dCCHUfO/EiROTPr8eFoDOt3Xr1iSO148aMWJE0ufXg7roootq7svnHV/L6euk4tjXOsTrTFXr37lzZ83j+nx2xBFHJPHtt99eafsc5c/Z12MB6Fg+F8Q13/6z7z+v48ePT+JDDz20xceNr6mk9Dpq5MiRSd+6desKzwPoyriSBwAAAICSuKECAAAAgJK4oQIAAACAknpcDdWECRMqbV8j5euetm/fnsS+PiHm146J152S0jVefK0WgMYT1x9J6bouvl7BO+qoo5L4gQceqLSL1rOTmtcrbdy4sdL2tVv1apvi8/T5zZs2bVoSx/UN/r1+PZlNmzYV7htA+xo1alQS+2uSmK8PPe+880of19dfxWtL+fqq9evXJzHXQuhOeEIFAAAAACVxQwUAAAAAJfW4IX8DBw6s2paaD8Xxj6P94+qYH+Lnpz6Oh8j4R/MAOp8fIuOH+O7du7fS9kPg/FC7Aw88MImLhtv5ocV+OOG2bdsqbZ87/HTIPo6nevfi30eSDj/88Jrn5bf1fxs/hCiO6w1xBLD//Odsw4YNlbbPbYsXL07ir33tazX366+D/DDjQw45JIlXrFhRaY8ZMybp83kk3hbo6nhCBQAAAAAlcUMFAAAAACVxQwUAAAAAJfW4Gqp4Sk9f9+THCvtxx348cGzq1KlJHE85LKU1B346ZgCdb+3atUns657i+iRfC+BrqHzuiGNfI+WXb/C1EHGtp69d8nlm7NixSRznO//7xH1S87ovXysRi6eQl5rXbq1cubLS9rVZANqeX8ogvs7wNY4+F/hlHmI+1/m8cPTRRyfx0qVLK+0hQ4YkfWvWrElivwwE0JXxhAoAAAAASuKGCgAAAABK4oYKAAAAAErqcTVUMV/n4NeZ8v1F4339GOTnnnsuiTdv3lxp+/oDAJ1v48aNSew///3796+57cEHH5zEvnYgXktq3LhxhcfxtZxxrZOvv/Q1VL4uKq7X8vVXW7ZsSWJfZxGfp9+vr6vwNRmrV6+utKmhAtrfsccem8QPP/xwpe3zhq/5Hj9+fM39FtVSStIFF1yQxNdcc02l7dfYi2srJWnkyJGF+wa6Ep5QAQAAAEBJ3FABAAAAQEncUAEAAABAST26hmrdunVJ7McV33nnnUl8xRVX1NzXSSedlMSzZs1K4okTJ1bavkYCQOfzayn59aDidV4WLVqU9B1xxBGF7/VrT8V8PZKvdYrPy6814+sxfa1EvG//+/maUb8uX1yT4eutfD2p37evxwLQvqZPn57E3//+9yttn3/imm5Juvfee5P43HPPrbR9faTnc99BBx1Uafv6K78vn1eArownVAAAAABQEjdUAAAAAFBSjx7y94c//CGJFy9enMR+yN+PfvSjmvs65phjktgPp/nv//7vSvv4449P+k4++eT6JwugXfkhwH6YTDxF+aZNm5I+/5les2ZNEsdDbPzwOD/E76WXXkrigQMH1jwnP6TGT6MeDy/u06dP0uenQn/22WeT+LDDDqu0//SnPxUexw/78UOKALQv/3mOP+9+CK7f1l/bxEP+ioYrS9Lo0aOTOJ4affny5UmfP494KQqgq+MJFQAAHW38eMms+k/BukAAgMbDDRUAAB1t1apyfQCAhsMNFQAAqODhGQC0To+roYqn7fTTFfsaKj+NetF4Xz/O2NdYxNOo79mzp2UnC6DDPProo0ns64TieJV7guCnEX/kkUeSOK6D8nVPPvZ5qW/fvpW2zx1+Wx/H06z7Kdd9zpo7d24SDx06tNL207P7v8327duTOP79L774YqFr4eFZ1xfXK/nPq7+W8cu87I946YbZs2cnfb5e1J8X0JXxhAoAAAAASuKGCgAAAABK4oYKAAAAAErqcTVU8Rowu3btSvr8eF5fc1DE78uPFY5rqnwfgM43aNCgJI5rASTp+eefr7S3bNmS9Pl1qHw90vDhwyttX2/kxXWeUroula+R8uvJDB48OInj+iu/rV8Pa9myZUl84YUXVtp/+7d/m/RdcsklSRzXiEnShAkTBKDznH766ZX2jTfemPSNHDkyieM8sb+mTJlSaW/YsCHp82vs+XwGdGU8oQIAAACAkrihAgAAAICSuKECAAAAgJJ6XA1VzI8b3rx5cxL7mooiffr0SWK/xktcNzWelRGBhvPud7+7sD9e12XJkiVJ32GHHZbEt956axLH61TF+5Gkl19+OYnjeitJWrt2baXtazV9nZdfpyqO/XpXY8eOTeKHHnooia+44opKe82aNUmfr9UqWqMPQMf74Ac/WGnPmDEj6fO5YOPGjUkc57dDDz20VccdMmRIpe1rTX2u8+v3AV0ZT6gAAAAAoCRuqAAAAACgpB495G/AgAFJ7IfPtGYYix8+6Kc+jh91t+UUpQA6RjzM7bjjjkv6/NCWdevWJXE8TbFfNsEPAfbTqsf78nnF5xI/lCeeprhe3vHHnTNnTqV9wQUXFL4XQGOZOHFipe2HEfthx34o8axZsyrt1g75i/OMH4Lsp033xwW6Mp5QAQAAAEBJ3FABAAAAQEncUAEAAABAST26hmrlypVJvHfv3iT2U3wW8dMI+1qGeN++dgtA4ymqg+zVq1fSN3PmzCT2yybEBg4cWHO/krR48eIkLqph8DnM7yuuC/XLQPg8FNdcSNL9999fafsaKv+3MbOa5wig/RV9Jt/whjckfbfccksS+/rK22+/vdK+9NJLW3Ue8bXQCy+8UHiOrbnGAhodT6gAAAAAoKQe/YQKAICWuOqqqwr7r7766g46EwBAo+EJFQAAAACU1KOfUI0bNy6JV69encS+TqLIiBEjkrhoPZixY8e2eL8AOoevCyrKB4sWLUpiv+5L/Pn39VX+vYccckgSx7VPzz//fM39Ss1rEnbs2FFp11vDyse+Pivm/zbUVAGdy3/243zlayBnzJiRxL6ecsWKFaXPY9iwYZW2X2fKXyetX7++9HGARsMTKgAAAAAoiRsqAAAAACiJGyoAAAAAKKlH11Cdf/75SfzII48kcWtqqIYMGZLE8ThiKV0PZvLkyS3eL4DGEK8l53PD8uXLk9jXNk2bNq3me4844ogkHjlyZBI/+eSTlbavTdq9e3cS+/qsOC/5nOTrG/w5b9++vWZfv379kpgaKqBz+brt2Gte85ok9mvObdy4MYnj+sm5c+cmfccff3zheQwdOrTSjnOIJPXp0yeJfa0p0JXxhAoAAAAASuKGCgAAAABK6tFD/vr375/E8bA8qXVD/rx4umIpffQ9adKk0vsF0DmKhrF96UtfSuKvfvWrSXznnXdW2n54jZ8m3Q/bi3OJX3Jhw4YNSbx58+aa/X4adD/cZvTo0Un8wQ9+sNL2Q/y8ouFGANpfa4bZHnzwwUk8Z86cJI6H5t1zzz1JX70hf1u2bKm0/XWQt2rVqsJ+oCvh/4IAAAAAUBI3VAAAAABQEjdUAAAAAFBSj66hete73pXEM2fOTGI/rXprXHjhhTX7jj322NL7BdA5iuqEBgwYkMSf/exna2777LPPJnE8LbrUvK4grot6+eWXC8/RT0scx75u4vTTT0/iwYMHF+4bQPfwqU99KonHjx+fxHHeOOuss1q17+nTp1fa48aNS/p83eY555zTqn0DjYwnVAAAAABQUo9+QgUAQFd31VVXFfZfffXVHXQmANAzcUMFAEAD4kYJALqGDruhmj179lozW95Rx2sLt912WxK/733va5P9/td//Veb7KcdTO7sEwDaUlfMOz0QeQfdRnfLOV/84hfbbd//8i//0m77roOcgzbXYTdUIYQxHXUsAJDIOwA6FjkH6JkY8gcAANoFwxYB9ATM8gcAAAAAJVkIobPPAShkZmskFY1JHy1pbQt21Rnb9ZRzm8xQF3QXLcg5Utf/zDbyMVuyHTkH3Uob5p1G/cx21jFbut3+XeuEENr8RwqjpDAn/1kpheejuG97HLOV5/dXUpgvhZelcIrr+4QUFkthkRTeGL1+Xv7aYilcFb1+gxQel8KXotc+LYW3Fhz/RClcK4V3R3+XXVJ4Im9f3Yl/mzFS+E1n/ztq3TnrkUbdriedW8P/SHuDNCdI84J0c5AG1tn+90E6JW8vC9LoDjzXa4M0N0iPB2lGkAZHfZcE6ckgzQ/SjflrrwjS7Hz70/LXegfpt4W/p/SNIJ2Zt98cpMfy4z4ZpCvy138QpIurvPfAIM2osd8pQfrrKD42SD/o9P8G2vCnO3xmG/WYrdmuc/8b4FqHa52OPu+u/ZntDudW66ddaqhC0DpJJ0iSmT4vaWsI+o+mfjP1DkF72uPY1ZipVwjaG700T9LbJX3HbXeUpEslHS3pQEm/NdO0vPt/JL1B0gpJfzbTHcpq0HaEoOPMdI+ZhkkaKOmVIejfCk7pk5L+LQTNlfT9/NjLJL02hPTuuMq5t5v838saM71optND0B874rhAB9mhEE6QJJndIOnvJX29U89IksxMkimEl6NXP6wQNuf9X5f0QUlXy2yqpE9IOl0hbJDZ2Hz7KyR9SNIySd+U9JeS3i/pxwphe43jjpL0KoVwpcz6SPqupFMVwgqZ9ZM0pfC8Q3hB0sVV9ts7f+9fS7ox3/YJmU2S2cEK4dnC/QJdBNc6XOsATTqshspMPzDTt830sKSvmOkEMz1kpsfNdJuZRuTb/d5Mp+Tt0fmHT2Y62kyzzDQnf8/U/PW/iV7/jpl65a9vNdPXzDRX0mnxuYSgBSFoUZXTvEjST0LQSyFoqaTFkk7NfxaHoCUhaJekn+Tb7pY0wEwHSOojaa+kf5H0uYK/wxBJx+UJptY2ybmb6SNmmpf/XJlvM8VM86L3fDRP6DLTP5npyfzv9JP8tUFmui7/Wz1mpovy1y830x1mulfS7/Ld/VzSO2udH9ANPCDpcJmdLbNfVl41+2+ZDGxBJwAAIABJREFUXV74TrOPyGxe/nNl/trVMvtAtM3nZfbRvP0xmf1ZZo/L7Av5a1NktkhmP1R20XNQcox9N1MmaYCkprHZ75X0PwphQ77d6vz13coucAZK2i2z4ZLeIumHBb/JX0r6Td4eouyiaV2+35cUQpwjz5TZn2S2RGYXR7/DvLx9uczukFlTHrla0hkymyOzD+f7+IWyizig2+Jap/J34FoHPUpHT0oxSdKrQ9BHlP2P/uMh6DhJT6jgg5n7e0nfDEEnSDpF0gozHSlpuqTT89f3at+HY5Ckh0PQ8SFoZgvPb6Kk56J4Rf5a1ddD0AJJayQ9quxi4XBJB4SgRwuOcYq0LznUUDl3STskvVvSKyW9StJ7zXRinfdfJenE/G/79/lrn5J0bwg6VdJrJX3VTIPyvpMkXRyCzsrjRySdUecYjeS7DbxdTzq3riF7gnK+srzT2veeLPd5lNmJkn4q6ZJoy0sk/VRm50qaquxC5QRJJ8vszHybqZK+pRCOVgjNx82bfV/SSklHSGpavG6apGky+6PMHpLZefnr/6Ps2+DrJX1J0mckfck99fJOlzRbkhTCekl3SFous5tk9k6Zxf9/mCDpNZLerOxmqZqTJF2sEM5SloMeUAgnKIT/zPu7Wl6ppzt8Zhv1mK3ZrhFxrcO1Tnvp6p/Z7nBuVXX0tOk3h6C9+ePi4SHoD/nr10u6uc57H5T0KTNNknRrCHraTOdIOlnZY2kp+ya36RvbvZJuaetfwAsh+xZFksz0C0lXmOlTko6XdE8I+j/3lgnKElOR+NxfI+m2ELQtP8atyhLAHQXvf1zSDWb6ubJvYCTpXEkXmumjedxf0sF5+54QtD56/2plwwC6hBBCiz4EnbFdTzq3LmCAzObk7QckXSvp1a3cx2sk3aYQtkmSzLLPYwjXyGyszA6UNEbSBoXwnMw+pOyz91j+/sHKbqSelbRcITxU80ghvFtmvZTdTE1XNmSmd/7+s5VdtN0vs2PzYXRn5+d0eN63QGY/ktRX0mcUwlPuCGkuCuHvZHaspNdL+qiyYT+X570/z2/OnpTZuBpnfE9+Y1ZLl8or9XSHz2yjHrM12zUornW41mkXXf0z2x3OrZaOvqHa1oJt9mjfk7P+TS+GoBvzR+hvkvRrM10hySRdH4I+UWU/O0uMx31e6dCbSflrKnhdkpQ/Vp6t7ILpsBB0iZnuMtMNISiuYdgR/141tOTc47+T3D7fJOlMZUN+PmWmY5X9rf7SP/430yvV/N9L//w8ge5kXw1VE7Oiz1Fr3ayspmi8sidWUva5+7JCSGoYZDZFLcmHIeyV2U8k/bOyG6oVkh5WCLslLZXZU8pusP4cveuLkj4t6Z8kfU9ZXdWX1HxoS/NcFMITkp7Ib8SWat8N1Uvx2dc423q/D3kFPQXXOlzroIfplHWoQtAmSRvMKo9aL5Mq3+AsU/ZNjBQVPJvpUElLQtA1km6XdJyycbAXm2lsvs1IM03ej1O7Q9KlZupnpkOUXajMUnaxMtVMh5ipr7I6gMq3JmbqI+lKSV9RWu/QS9m3w7EFyh6Xt9QDkt5qpoH5Y+u35a+tkjTWTKPM1E/ZUBzlY5wPCkH3Sfq4pGHKEt9dkv7RLLsYqvMofZrqP6oHuoPlko6SWb+87uicOts/IOmtMhsos/jzKGU3UZcqy1tN30LfJek9MhssSTKbGE0kUZ2Z5U+ZmmqoLpS0MO/9ufY9iRqt7LO6JHrvWZJeUAhPK6unejn/GVjlSPtykdlgmZ0d9Z2g+tP3FtmirC4rRl5Bj8K1Dtc66Dk6c2Hf/6dsbOvjyv7n/S/56/8h6f1mekzZnPBNLpE0z0xzJB0j6Ych6Ell38Tene/nHmWPmQuZ6W1mWqGsgPNXZrpLkkLQfEk/k/SksmLtD4SgvfksPR9U9kFdIOln+bZNPqDs26Ptyh5BDzTTE5Jmh6CN8bFD0EJJw/KCzbryMco/UJbsHpb0vRD0WAjarexvNiv/vZsuuHpJ+nF+/MckXZOfw78qKyZ93Ezz87iW10r6VUvOr7OZ2XlmtsjMFpvZVTW2uc7MVltTAX3tfR1kZveZ2ZNmNt+y4Vp+m/5mNsvM5ubbfKHOPnuZ2WMWT3zQfJtlZvaEmc0xs0dqbDPczGaY2UIzW2Bmp1XZ5hX5Ppp+NlvTpAnNt/1wfv7zzOwmM6v6TaKZfSjfZn6tfXVpITyn7DM/L//nY3W2b/Z5VAiP5X3zld1EPK8QXsxfu1vZTHcPyuwJSTPU/EbDM0nX59s/oSynNeXHuyStk9mTku6T9DGFkE0kkd18fVr7PtvfVTbj36+kfTOPRX6lppuz7Jj/nE+UMUfSF7Tv6VQZj0vaK7O50aQUXSav1NNWeaclOSffrsV5p61yTr5dm+WdHpxzuNZpAa51ivWUa518u8K807DXOvsz5zo/5X6k8GEp/F1nn0fB+d0vhRGdfR71z1O9JD0j6VBl347NlXRUle3OVFaMOq/O/iZIOilvD5H0lN+fsgvPwXm7j7LE/6qCfX5E2QX1Lwu2WaY6axwpG3v/d3m7r6ThLfjbrFS2CJ3vm6hsONeAPP6ZpMurbHeMshuNgcqGB/9W0uGd/e+dnzb8kWaGOv8ttdFx+gXpoSD17vTfeT9/2jLvtCTn5H0tzjttlXPy7dok75BzeuYP1zptdZ4951on367FeaeRrnU68wlVT/a/SmsSGoaZxkj6egja0Nnn0gL5FK9hSQghnuI1EUK4X1JRsXzTdi+G7AmEQghblH1DN9FtE0IIW/OwT/4TVIWZTVI2xvt7Lf6Nqu9nmLJEeW1+DrtCCBuL36VzJD0Tqs0el+ktaYBlM94NlPRClW2OlPRwCGF7CGGPsqEqby/zO6Bh/X/aV7Ddng6WdJWy/466ujbLOy3JOXlfi/JOW+WcfF9tnXfIOT0P1zpto0dc6+T7am3eaZhrHW6oOkEI2hmCftTZ51FNCFoTQmW2nEZXa+rX/WbZpAEnKvtWxvf1smxY1GpJ94QQmm2T+4ayyQSKpq6WsiR1t5nNNrP3Vek/RNlsSd/PH6l/z7L6nSKXSrqp6sFCeF7ZcJNnJb0oaVPIhqZ58ySdYWajzGygpAvk10tC1xbCwwrh8Q44ztMK4fftfpyO0S55pyjn5P0tyTttlXOkNsw75JyeiWudNtNTrnWk1uedhrnW4YYKcCybQOAWSVeGpgVWIyGEvSGbLW6SpFPN7Jgq+3izpNUhhNktOORrQggnKVsb6QO2b52iJr2VPcb/3xDCicpmKqo6hjo/dl9lExlUnZ7XzEYo+3brEGVTxg4ys7/x24UQFkj6d0l3KxtnP0fqmJXsgZ6kXs6R6uedNs45UhvmHXIO0Hga8FpHakXeabRrHW6o0JUVTf1aipn1UZZgbggh3Fq0bf4Y+j5J51XpPl3ShWa2TNnj+deZ2Y9r7Of5/J+rJd2m7PF+bIWkFdG3QzOUJZxazpf0aAhhVY3+10taGkJYE7Lpt29VjfWYQgjXhhBODiGcKWmDsrHWQE/WpnmnNTlHKsw7bZlzpLbNO+QcoLyecq0jtS7vNNS1DjdU6MryKV7tkPybimSK19YyM1M2bndBCOHrNbYZY9n02jKzAcoWP13otwshfCKEMCmEMCU/r3tDCM2+GTGzQWY2pKmtbFHCeW5fKyU9Z2avyF86R9nsTLW8QzUegeeelfQqMxuY/87nKBtDXe33zafptYOVjSm+sWC/QE/QZnmnJTkn365u3mnLnJPvry3zDjkHKK9HXOvk+2tN3mmoa52OXtgXaDMhhD1m1jTFay9J14Vs6uqEmd2kbHro0Wa2QtLnQgjXVtnl6crWCXkiHzcsSZ8MIfw62maCpOvNrJeyLyR+FkKoOU1oC4yTdFv2WVdvSTeGEH5TZbt/lHRDnkyXSHp3tZ3lieoNkq6odcAQwsNmNkPSo8oWTXxM2RTb1dxiZqMk/f/s3Xe8XVWZ//Hvk0p6bwQwCSkQEkISwFCkCQgIKMoMilJ0GGQYpegg/ByUUUcBe0FgUBhEpEoRFKmhOVLTExIgCQmEdNILqev3x17ZWWvlnHNvdm5u/bxfr/tyPWftc/Y+B8+Tvc9ez1qbJP17NYrSgUathvNOdXKOVLN5p7o5R6qhvEPOAYprYuc6UjXyTn081zHnSk7aAQAAAACoAkP+AAAAAKAgLqgAAAAAoCAuqAAAAACgIC6oAAAAAKAgLqgAAAAAoCAuqAAAAACgIC6oAAAAAKAgLqgAAAAAoCAuqAAAAACgIC6oAAAAAKAgLqgAAAAAoKAWdX0AQFW6d+/u+vXrV9eHgQrGjRu31DnXo66PA6gJ5Jz6j5yDxqYx5p1Jk6TNm8v3t2ghjRhRe8ezqyrlHS6oUO/169dPr7/+el0fBiows7l1fQxATSHn1H/kHDQ2jTHvmFXu37xZakhvuVLeYcgfAAAAABTEBRUAAAAAFMQFFQAAAAAUVGs1VA2h2G7Lli1R3Lx58yjesGFD3t6cVNlZMlA0jdu0aVMTh7hbUeRbM3r3lhYtKt3Xq5e0cGHtHk9T1hDyTuqDDz6I4rVr1+Zt51zUl+aoPfbYI4q7d+9ew0dX88g7aEwaYs5pasg52B1q7YKqrort0hOQ9EIntHz58iju0qVLFM+aNStvL126NOpLT2xat24dxcOHD6/6YOsYRb41o9zFVFV9qHl1lXe2bt0axWEeSnNF6o477ojil156KW+nP+SkOWq//faL4i9/+ctl97MzubEmn5si76AxaYwTCzQ25BzsDgz5AwAAAICCuKACAAAAgIIa3TpUVdVBhUNV0mF5mzZtiuK07mn9+vV5u3PnzhWf27Jlyyj+13/917z9ox/9qOSxA2gcmjWr/m9VkydPjuLzzjsvig877LCyr5vmmZ///OdlXyvNhekwvZ0ZxrcrQ/wAAGhsuEMFAAAAAAVxQQUAAAAABTW6IX9VzaB177335u3vfOc7UV869Ob++++P4iuuuCJvT5gwIep7+umno/j444+P4osvvjhvpzN1tWgR/2eoyRm0ANS9GTNm5O1FyVSPPXv2jOJXXnkliq+55pq8vXLlyqgvHZb8u9/9LopfeOGFvP33v/896rvyyiujuFWrViWPHQAAVMYdKgAAAAAoiAsqAAAAACiICyoAAAAAKKjR1VBVJaxX2nPPPaO+q6++OopPOeWUKH788cfz9jvvvFNxPzfeeGMU9+vXr9rHSM0U0LCMGzcuih9++OEonj9/ft4+4ogjor4VK1ZEcdeuXaN4yJAheXvx4sVRX1pDNWLEiCjeuHFj3u7YsWPUly7fcPTRR0fx/vvvn7e7d+8uAABQGneoAAAAAKAgLqgAAAAAoCAuqAAAAACgoAZRQ1VpXaawRkCSxo8fH8VpfcKHH36Yt2fOnBn1TZ06NYofe+yxKO7cuXPe7tOnT9T31ltvlTz2bd588828vWHDhqgvreXatGlTFPfq1StvN2vGNTBQ19I1nD7+8Y9HcVpzFNZBDRs2LOqbM2dOFP/hD3+I4tGjR+ftwYMHR31pLnnkkUei+BOf+ETeDmuiJOnll1+O4nQtvbD/05/+dNQ3aNAgAQCADGfnAAAAAFAQF1QAAAAAUBAXVAAAINe7t2RW+q9377o+OgCofxpEDVWldZneeOONKH7ttdeiOKxdkOIahIMOOijqe//996N4zZo1URyuLTNy5Miob+nSpVG8fv36KG7Xrl3e/uCDD6K+t99+O4pbtWoVxS1btszbrAcD1I0pU6bk7bRW6frrr4/idN25cP27AQMGVNx2+fLlUfylL30pb8+ePTvqW7duXRRPnDgxij/60Y+W3Tat3ezbt2/Z1/rZz34W9d10001C47VoUbE+AGiquEMFAAAAAAVxQQUAAAAABTWIIX+VpMNjBg4cGMXpsL0ePXrk7VWrVkV93bp1i+J0eN3rr7+et1999dWoL50KecmSJVG8evXqvN2lS5eK+02nRk+HDwKofePGjcvbjz/+eNR32223RfGf//znKA6/4+n05TNmzIjiRx99NIrDPJVOsb4oGX+VDh8Ol1wIl26Qdhw+2LVr1ygeOnRo3v7kJz8pAABQGneoAAAAAKAgLqgAAAAAoCAuqAAAAACgoAZZQxXWRYW1SZLUp0+fKE6nNx4+fHje/vDDDyvup3379lG8cePGvJ3WNYVTm0vSli1bojic+r1t27ZRXxqn0xunMYDaN3bs2Lzdv3//qC9dgqFTp05RHOaStN5y7ty5UZzmsOOOOy5vz5o1K+rbtGlTFIdTu0txHWhabxXWV5V6rdC8efOiOF0mguUcAABNGXeoAAAAAKAgLqgAAAAAoCAuqAAAAACgoAZZQ7VixYq8vWHDhqivd+/eUZzWDYTrQ7Vr1y7qa968eRTvscceUdyxY8e8ndZMOeeiOF1bKqyp2Lp1a9SXxmGtlhTXNqTvt3Xr1gKw+4XrQb333ntR38EHHxzFaR1UWK/ZuXPnqC9dSy/NLYMGDcrbK1eujPrS+st0ramwxjTdb5rfjj766Ch+4IEH8na6vtUHH3wQxdRQAQCaMu5QAQAAAEBBXFABAAAAQEFcUAEAAABAQQ2+hqpVq1ZRX1on0KVLlygOa5DSvrQOqlmz+HozrINo06ZN1JfWI6RrXIXrVqV1D2md1+bNm6M4fE9hHYck9ejRQwB2v/B7mNY9PfbYY1Gcfi/D739a5zlnzpxqxzNmzIj6unbtGsWzZ8+O4gsuuCBvz58/P+qbOHFiFD///PNR/I9//CNvpzkrreUEAKAp4w4VAAAAABTEBRUAAAAAFNQgh/yFw2fSIX/p1OfhtpK0dOnSvJ0Oy0mH+JlZ2WNo0SL+6LZs2RLF6VTo4fTm6XPT4YJpf6VtAdSO0aNH5+3zzjsv6guHx0k7Dr1btmxZ3l6wYEHUlw4fXLNmTRSHQ5zDadClHfNOOp35vHnz8nY69fm6deuiOMyNUjwVfDqUOh1qCABAU8YdKgAAAAAoiAsqAAAAACiICyoAAAAAKKhB1lCFU5KnNVNpjVE6JXlYr5DWBaT1Bxs3bozisLYp3W9ay5XWY4U1VR07doz60umKR44cGcVhLVc6tTuA3WPKlClRfPfdd+ftz3/+81FfWjOZLn3QqVOnvN2+ffuyfdKOeSeMN23aVPGYu3XrVva109rMNEelOeykk07K2wsXLoz6nn322Sg+55xzKh4XgJqVnq+kdZthjeS7774b9Q0bNiyKb7nlligOv8977rln1Jfmq3T5mVCaF9OcU0l6rlOpph2oD7hDBQAAAAAFcUEFAAAAAAVxQQUAAAAABTXIGqqwpqBt27ZRXzrudtWqVVHcu3fvvB2u7yLtOEY3He8b1iCkY4PT57Zs2TKK05qK0J/+9KcoHjx4cBSHY5jD+jEAu8/atWujOKwjuv3226O+xx57LIqvueaaKA6/07169Yr60rqo999/P4oPO+ywvJ3mpJ49e0Zxuj7UoEGDym6brn91xhlnRPH06dPz9qRJk6K+UaNGRTE1VMB25Wqdq6oDSteVC2u1x44dG/X9+te/juJZs2ZFcZi/0vrIfffdN4rTWvOjjz46b99www1R39NPPx3FjzzySBSPGTMmb1dVM5XWi4bHSc0UGhruUAEAAABAQVxQAQAAAEBBXFABAAAAQEENsoZqw4YNeTtdAyEduzxjxowoDtepat26ddS3fv36KE7HM1fqq7TulLTj2jOhhx56KIq/8Y1vRHE4rnjNmjVlXwdAzRk6dGgUX3vttXn7xBNPjPp69OgRxQ888EAUh2u37LXXXlFfmjvuuuuuKB4wYEDeTuskFixYEMUvvvhiFIf58b333ov6Vq9erUpOOeWUvH3sscdGfelnA6C08FygqpqidH3L8ePH5+1f/OIXUd+QIUOi+Kyzzori0aNH5+10zc205vOll16K4t/+9rd5u0OHDlFfWgOa1l72798/b1911VVR3+mnnx7FaW0X0JBxhwoAAAAACuKCCgAAAAAKapBD/sLpNDt27Bj1hcMBJWnOnDlRHN6+TrdNpyRPpz4Pb9ent+7TW/WpcHr3dFhiOJW7tOO0yQceeGDeTocSAtg93n777Sh+66238nb6fV+8eHEUp8skhEOE06HF6WulQ/OmTZuWt9MhzGkOS3NLOCX7u+++G/UtW7Ysig844IAoDof2pJ/F5MmTozjMUUBTF56jVHVuUEk4bO+DDz6I+tIlEnbGeeedVzEOvfPOO1H83//931E8ceLEKA7LEsJh0qVeq0+fPlEc5qQ0l6VlFum5ULh9uhTFcccdJ2B34w4VAAAAABTEBRUAAAAAFMQFFQAAAAAU1CBqqNLapnAsbTr1+apVqyq+1rp16/J2u3btor4WLeKPI62hSsflhtKaiXAMtRRPD5rWSM2fPz+K582bV3Y/1FABtSOtGwqXXEhzwX333RfF1113XRSH9UnpFMbpdzqst5Sks88+O29PmDCh7DFJO9YonHzyyXn7sMMOi/rSGqrLL788isN9hXlT2jE3rlixIorT9wg0FRs3boz+DQ9rL9Pvdps2baI4rc2+7LLL8nZaL/mPf/wjitPvYHjelOartB7p1VdfjeKFCxfm7bROfb/99oviE044IYoHDRqUt9MlIh5++OEoTpd5CGvc0xyT5sn0nCvsTz+LQw45RMDuxh0qAAAAACiICyoAAAAAKIgLKgAAAAAoqEHUUKXjfUPp+N5wvHIp4ZjltP4q3U+4noIUryeRjudNxz5XGu/bt2/fqC9c70XasXYjlNZmpce8K2teANhu3LhxURyu+5KuCfPmm29GcVqPOXbs2Lw9ZMiQqC/NM88//3wUjxw5Mm+n+S2tFUiP66ijjsrbL730UtQX1nVK0j777BPFYQ1VmrOWLl0axUuWLIliaqjQVDVv3lzt27fP47CWKV0LLq2XTs8jhg8fnrdvvfXWivtNa6zC73daL96zZ88o/ud//uco7t+/f95O14raFV/5yleiOK15D/NmWiOVStepSuMQ+Qi1gTtUAAAAAFAQF1QAAAAAUBAXVAAAAABQUIOooUqF44zTdR3Gjx9f8blhDdX69eujvrT+KF0HoVJ9UlozkY6FrjS+NxxvLe1YjxGqai0GaqiAmpGu2zRmzJi8PXXq1KjvyCOPjOIuXbpE8ZQpU/L2xo0bo770O53mirBOMs13ae1SWlMZ5od0LZq0hirNQ2F9Q1jLIUmrV6+O4rQmA2iqmjdvHtXsnHLKKXV4NPVbWgMPNGTcoQIAAACAgrigAgAAAICCGsSQv3SoSjhEJp1GfNmyZRVfq0OHDnl77dq1UV86FCcdPhMOkalqSs906F04vDAdStitW7coTt9vaGeGEgIobuLEiVE8cODAsn3ptOILFiyI4vfffz9vp9MQp8PnKk2t/M4775Ttk6R169ZF8aJFi8q+bpp3Bg8eHMVhPtxrr72ivrlz50bx8uXLo7hTp04CAKCp4A4VAAAAABTEBRUAAAAAFMQFFQAAAAAU1CBqqCpNM5xOV55OhZ4Kx/YvXLgw6kvrk9asWRPFGzZsKLttWstVqe4rnSo0rTdIax1C6ftNp1wGUDP+8pe/RHFYr/jLX/4y6vvEJz4RxaNHj47iMF+MGjUq6nvvvfei+NBDD43iAw44IG+n3/c0d6S1nSNGjMjbaX1pOrV7OgX717/+9bydLuUQ1oRJ0re+9a0o7tevnwAAaCq4QwUAAAAABXFBBQAAAAAFcUEFAAAAAAU1iBqqdK2ltF4plK7/MmjQoLLPTdeKSuuTKsXpGlVVrQeV1lyF9t9//yieMWNG2W2poQJqx09+8pMoPuyww/J2Wl+57777RvGKFSuiOKyp3GOPPaK+zp07R3Hv3r2jOFzjKv2+z58/P4pXrVoVxWG+23vvvaO+Dz/8MIrTWtULLrggbx955JFRX3ocaT8AAE0Jd6gAAAAAoCAuqAAAAACgIC6oAAAAAKCgRldDla7htNdee5V9rXBdKWnHuqh0TauwbiA9hrSmIO2vtD5W+/btKx5HGKd1X+m6MwBqxuzZs6M4rH1Kv6NDhgyJ4meeeSaKH3zwwbw9fvz4qC+tg7r99tujePny5Xk7XbNq+vTpUZzWRYWvPXHixKjvgw8+iOITTzwxisN1qRYtWhT1pfVWac1Yjx49BABAU8EdKgAAAAAoiAsqAAAAACioQQz5S4VTEKfSoXUDBw6M4nDIXOvWraO+dJheOrwu7E+HvKTS51bSrl27KE7fw7p16/J2Om16VccBoJi1a9dGcTgELmxL0sEHHxzFo0aNiuJw+YZ0ivFJkyZFcTqt+uc+97m8PW3atIr7SYcinn322WWPcdmyZVF80kknRXG4r3Sa+PSzqTSkGQCAxo47VAAAAABQEBdUAAAAAFAQF1QAAAAAUFCDqKFKpwKuVJ80Z86cKD788MOj+J133snbCxYsiPratGkTxV26dInisHYrrVVIpy9P67wq1X2l+125cmUUh/tKa6gA7B6rV6+O4nDK8pkzZ0Z9bdu2jeInnngiisPvcJorFi5cGMVDhw4te0zpfoYPHx7F6VTvnTt3zts9e/aM+tKp0NN8GC7nkC5HkX42aY4GAKAp4Q4VAAAAABTE7Q4AABqwq666qmL/ddddV0tHAgBNE3eoAAAAAKCgBnGHKq05CNdpSWuZ0rH86dorzrm83apVq6gvfa10nZawpmDr1q1RX7ouS1pj0KzZ9mvX9BjTtWR69+4dxWHtxpAhQ6K+SrVZAIpL65PGjBmTt996662or2XLllG8atWqKA5zTVoj+dJLL0Vx9+7do/jpp5/O2+l6UAMGDIjiV155JYpPOOGEvB3mEWnHetPBgwdH8dFHH52333jjjaivY8eOUbzvvvsKAICmijtUAAAAAFAQF1QAAAAAUBAXVAAAAABQUIOooTKzsvH8+fOjvo0bN0bxmWeeufuUjsF/AAAgAElEQVQOrIJu3bpVe9u0ziutkxg7dmzeTus60totADVjn332ieJnnnkmb6frMoU1kpI0efLkKN5zzz3z9rp166K+tJapa9euZY8prSddv359xTis5Uz3m9ZUhfWlktS6deu8na5Z1bdv3yhO1+wDAKAp4Q4VAAAAABTEBRUAAAAAFNQghvzNnTs3isNph1esWBH1ffvb366VY9qdLr300iju379/3l64cGHUl07fztAboGakw2t//etf5+1XX3214nPPPffcKH755ZfzdvPmzaO+dIhvOlx41qxZeTudnj0dxpfG4VDEdDh0miv222+/KA6HLaZDGPv16xfF6bBsAACaEu5QAQAAAEBBXFABAAAAQEFcUAEAAABAQQ2ihqp9+/ZRvGnTprzdsWPHqO+YY46p9uum0wTXlzqAz372s1HcqlWrvL1ly5baPhygSWrRIk6Pn/nMZ/J27969Kz532LBhFePQl7/85SgePXp0FIf5Lpx+XdqxlqlPnz5RPHTo0LLbnnbaaWWPKT2OdFr4vffeO4rrS+4EAKAucIcKAAAAAAriggoAAAAACuKCCgAAAAAKqrUaqnHjxi01s7lVb4k69JG6PoC6cNVVV1Xsv+6662rpSFDTaivvXHvttbt7F41Zk8w7aJw412kQyDmocbV2QeWc61Fb+wIAibwDoHaRc4CmiSF/AAAAAFBQg5g2HQCAxqTSUGOGGQNAw2LpWkxAfWNmSyRVGpPeXdLSarxUXWzXVI7tIwx1QWNRjZwjNfzvbH3eZ3W2I+egUanBvFNfv7N1tc/qbrdr5zrOuXr7J7nekrtHcrMkN05yj0lucIHX6Sy5iyv0Xyq5qZKbJrnLkr6vSW6G7/uRf+wIyU2W3OuSGxTs40nJNauwnz9JboBvt5fc/wTv7TnJfbTg5/StoN1Kci9IrkVd//ervf+f6PX6ul1TOrbG8Ce5//Tf9cmSm7jtOym5OZLrXmL70yV3VZnXOkZyh5fp6yS5RyU3ye/vS0HfeZJ72/+d5x9rLbnHfZ66ONj2FsmNqvB+Pi257/j2EJ9nJkpuuuRu8Y+fL7kbyjz/Mcl1LtNH3qnl7ZrKPndmu/ryVy531MDrPie5g4tsI7mvSm6m5FyYvyRnkvuV75sc5pDazj818PkcI7m/VHcbyZ0que/V9f9fir/fhv2dbQzHVu6v3tZQmckkPSTpOee0r3MaLen/SepV4OU6S7q4zH6GSfpXSYdKGiHpVDMN9H3HSvqUpBHO6QBJP/FP+4akUyRdJuki/9jVkn7onLaW2c8Bkpo7p9n+od9JWiZpkH9vX1J2dVzEt7Y1nNNGSc9IOqvgawFNkpkOk3SqpFHO6UBJx0t6r9JznNMjzmmH8VlmaiHpGEmHl3nqv0t6wzmN8Nv91EytzNRV0jWSPqosJ11jpi6SPiHp75IOlHSO38cIZTllfIVD/KakG337V5J+7pwOck77S/p1pffm398pzmlF8t7MTM1E3gEkFcsdteT/lB1LetfjZEmD/N+Fkm6SpPqWf3aTv0o6zUxt62j/aKTq7QWVpGMlbXJON297wDlNck4v+n/Qf2ymqWaaYpb9I26m9mZ6xkzj/eOf8k+9TtK+Zppoph8n+9lf0ivOaZ1z2izpeUmf8X3/Juk657TB73+xf3yTpLb+b5OZ9pW0t3N6rsL7+YKkP/vj3FdZwrp62wWYc3rHOf3V93/dv7epZrps2wuY6WEzjTPTNDNd6B+7TlIb/97+6Dd92O8PQPX1kbQ0+L4vdU7zg/6vBbllP0ky0/lmusG3bzfTzWZ6RdJ9yn5sudx/Nz+W7MtJ6uB/OGqv7MeVzcpOXJ5yTsuc03JJT0k6SdtzTktJ5l/j+5K+Xe7NmGmwpA3O5UMY+kialx+A05Rg8z3N9LiZ3jbTj4LXmGOm7mbqZ6Y3zXSHpKmSbhV5B9imbO4w03fM9Jr/9/wW/52XmZ4z0/VmetVMb23LEWZqY6Z7zDTdTA9JarNtJ2a6yUyv+3OA71Z1UM5pgnOaU6LrU5Lu8D+svyyps5n6qA7yj88tL/rcOt4s+xHKTMf4z+hPZpphpj8Gn91J/rHx2n6+JjMdaqaXzDTBTP8w05ASn4mT9JyyC2CgxtTnC6phksaV6fuMpIOU3VE6XtKPfTL4UNIZzmmUsguyn/ov4FWSZvlfRq5IXmuqpI+ZqZv/xeIUSXv7vsG+7xUzPW+mQ/zj10q6Q9kdsxsk/UDZHapKjgjezwGSJjqnLelGZvndqo9KGiPpX8000nd/2d/NOljSJWbq5pyukrTev7dtJzNTpfxYm4Jb6vF2TenYGronJe3tT25uNNPRSf9Sn1tukvQfZV5jL0mHO6fPSLpZ23+RfTHZ7gZlP+bMlzRF0qX+x5W+in/Znucfe0pSP0kvS/qVmU6XND654EsdIUW/Hv9c0lgz/c1Ml5upc9B3kLK7S8MlnWWW58DQIEk3OqcDnNOXRN5pDN/Z+rrPndmuPqiUO25wToc4p2HKLo7CE/kWzulQZaNdrvGP/Zukdf4uzjWSRgfb/6dzOljZnaKjzXRgweMtl2fqIv8slnSCz61nKbuTtc1IZZ/NUEkDJB1hpj0k/VbSaco+m97B9jMkfcw5jZT0HUk/LHNsr0s7/MjVUDT072xjOLaS6vMFVSVHSrrbOW1xTouU3VU6RNkvJz8002RJTytLBBWHCDqn6ZKuV5YQH5c0UcovdFpI6qrswuYKSfeZyZzTROc0xjkdq+xLvkCSmeleM91pVnKffSQtqeZ7e8g5rXVOayQ9qO1f/EvMNElZUttb2QlOqfe0RdJGM3Woxv4aPOdctb4EdbFdUzq2hs5/30YrGwKzRNK9Zjo/2ORB/7/jlJ1clHJ/qR9KSviEslyzp7KLmRvM1LHCsW12Tmf7E4X7lZ1k/NRMP/O/4J5e4mlRznFO/6vsIu5+ZcMMXzZTa9/9jHNa6Zw+lPSGSi98Odf/ml3uGMk7u3m7prLPndmuPqgidxzrf5SdIuk4ZT+oblMqpxwl6U7/upMlTQ62/2d/V2aCf52hNf5mStjN+aelpN/6z+d+xe/pVec0z//YNFHZZ7SfpHec09v+btOdwfadJN1vpqnKLuDCzzq0WFnubXAa+ne2MRxbOfX5gmqa4l9mquMLknpIGu2cDpK0SNIeVT3JOd3qnEY7p6MkLZf0lu+aJ+lBf1v8VUlbFdQ5+btfVyu79X2NsvHCv5V0SYndrA+OZZqkEWZqXt03ZqZjlN2NO8zXXUyo4r21VnbHDkA1+R9pnnNO10j6qqTPBt0b/P9uUfklJ9ZWc1df0vbcMlPSO8pOFN6XortDe/nHQhcru0M+RtJKZb/qfqPEPsKcI0lyTvOd023O6VPKhhgO810bgs3Kvb/qvDfyDpqkUrnD3025UdKZzmm4svOD8DtZnZwiSTJTf2V3xj/u67T+qmqc35RRLs/URf65XNm52ghlo29aBU+pTl4KfV/Ss/5u4Gnp/gN7+OMDakx9vqAaK6n1tlohSTLTgX6c8YvKhqU0N1MPZb/ovKrs14nFzmmTZRNKbPuVdbVU/ldTM/X0/7uPsuGEd/muh5UNHdw2HriV4ikVz5X0mHNapmx88Vb/V6rYcbqUTXbhnGYpu+X83WBMcD8zfdK/t0+bqa2Z2kk6wz/WSdJy57TOsvqNMcFrbzJTy+D9dFM2PGlTufcMIGamIWbRXd+DVPUUtpVUyjvvSvq4328vSUMkzZb0hKQTzdTFsmLwE/1j246xi7IhQ3doe85xCuosAnnO8c89aVueMFNvSd2048nSziDvAKqYO7ad0C81U3tJZ1bj5V6QdLZ/3WFSPqyvo7IfNVb6nHHyLhzyI5LOtawefYyklc5pgeom/3SStMDfhTpHqvKH5hmS+llWiy5Jnw/6Oml7Tju/wmsMVjZEGagx9faCyt/KPUPS8WaaZaZpymqXFiqb/W+ypEnKLry+6ZwWSvqjpIP9reNzlX3x5Jw+kPR/lhWFppNSSNIDZnpD0qOS/j2Y1eo2SQP87eN7JJ3nj0u+3up8Sb/x2/5M0mOSfiFtn0gj8Fdlt7m3uUDZcMSZ/vVvV3YxON63X5X0iqTfOacJyoYjtjDTdGWTbIRDb26RNNm2F4cf6/fX6JnZSWb2ppnNNLOSK2Wa2W1mttjMKiZQM9vbzJ41szfMbJqZXVpimz3M7FUzm+S3qVgYbGbNzWyCmf2lwjZzzGyKmU00s9fLbNPZzP5kZjPMbLqZHVZimyH+Nbb9rTKzy8q83uX++Kea2d1mVvKXPDO71G8zrdxrNSLtJf3eTG/4YcNDJf3XLrzeo5LOsNKTUnxf0uE+Vz0j6UpfyL7M973m/77nH9vmO5J+4E8+nlA2HHiKpD+U2P8LkkZu+9FG2cnRVD9s+AlJV/i8WRR5ZxfzTnVyjt+u2nmnpnKO367G8k4jzzklc4c/l/itspP3J5R9p6tyk6T2/t/678nXXjunScpGpsxQ9qPv/1X1Qma6xEzzlN1pmmym3/mux5T9gDPTH9/Ffh91kX9ulHSef3w/VXEn3A9LvlDSX/3wx8VB948kXWumCap8N6vB5aqmcq7jt6uYd+rtuc6uzLnOX/X/JNdGci9Lrnkt7OtBFVivq6H9Kfsla5ayOrZWyi6wh5bY7ihJoyRNreL1+kga5dsdlA39HJpsY5La+3ZLZRe9Yyq85teV/eNXdp0MSXMk7bDGUbLN7yVd4NutJJVcGyj5bBYqW4Qu7eurbIhZGx/fJ+n8EtsNU3Yi0FbZP05PSxpY1//d+av+n+R+Kbnja2lf5J14uyrzTnVyju+rdt6pqZzjt6uRvEPOaZp/tZl/duKYeknumbo+jp075qZzruO3q3beqU/nOvX2DlVj45zWK6uz6rs792OmVpIedi6vA2vMDpU00zk32zm3UdldxE+lGznnXpCiX9lKcs4tcM6N9+3VyoYs9E22cc65NT5s6f9cqdczs70kfVLKfxUsxMw6KUuUt/pj2OicW1H5Wfq4pFnOuXJD1lpIamNmLZQlkVKzNfklBdw651y6pAAahh+q9BDkGkXeKZZ3qpNzfF+18k5N5Rz/WjWdd8g5TU+t5J+dtI9K13zVZ03iXMe/1s7mnXpzrsMFVS1yTk84p3d38z42Oqc7duc+6pFyU7zuMjPrp2zK1ldK9DU3s4nKhho85ZzbYRvvF8omKim52HPASXrSzMaZ2YUl+vsrmy3pf/0t9d+ZWbsqXvNzku4uuTPn3le2SPW7ymaoXOmce7LEpn5JAetmZumSAmgAnNMi5/RILeyHvLOLKuUc31+dvFNTOUeqwbxDzmmaaiv/7Azn9JpzmljXx7GTmsq5jrTzeafenOtwQQUkzKy9pAckXeacW5X2O+e2OOcOUjYu/VAzG1biNU6VtNg5V24ttdCRzrlRyoqM/93Mjkr6Wyi7jX+Tc26ksjHmJcdQ+323knS6siloS/V3UfbrVn9lU8e2M7Mvpts55yotKQCghlSVc6Sq804N5xypBvMOOQeof+rhuY60E3mnvp3rcEGFhqw6U7zuFDNrqSzB/NE592Clbf1t6GeVrSSfOkLS6WY2R9nt+ePM7M4S2237FUXOucXKJlw5NNlknqR5wa9Df1KWcMo5WdJ459yiMv3HS3rHObfEObdJ2Vooh5c5tludc6Odc+mSAkBTVaN5Z2dyjlQx79RkzpFqNu+Qc4Dimsq5jrRzeadenetwQYWG7DVJg8ysv/+l4nNS8eEFZmbKxu1Od879rMw2Pcyss2+3kXSC/GySIefc/3PO7eWc6+ePa6xzbodfRsysnZl12NaWnwkpea2Fkt4zsyH+oY8rW3y1nM+rzC1w711JY8ysrX/PH1c2hrrU+/VLCli6pADQVNVY3qlOzvHbVZl3ajLn+NerybxDzgGKaxLnOv71dibv1KtznaoWSQPqLefcZjP7qrIpWJtLus05Ny3dzszuVjZlfXczmyfpGufcrSVe8ghl62BM8eOGJelbzrnHgm36SPq9mTVX9oPEfc65stOEVkMvSQ9l33W1kHSXc+7xEtt9TdIffTKdrWxh2B34RHWCpK+U26Fz7hUz+5Ok8coWV5ygbArsUh4ws26SNkn692oUpQONWg3nnerkHKlm8051c45UQ3mHnAMU18TOdaRq5J36eK5jzpWctAMAAAAAUAWG/AEAAABAQVxQAQAAAEBBXFABAAAAQEFcUAEAAABAQVxQAQAAAEBBXFABAAAAQEFcUAEAAABAQVxQAQAAAEBBXFABAAAAQEFcUAEAAABAQVxQAQAAAEBBLer6AICqdO/e3fXr16+uD6NemzRJ2ry5fH+LFtKIEbtv/+PGjVvqnOux+/YA1B5yTv1HzkFj0xjzTl2fm9S0SnmHCyrUe/369dPrr79e14dRr5lV7t+8WdqdH6GZzd19rw7ULnJO/UfOQWPTGPNOXZ+b1LRKeYchfwAAAABQEBdUAAAAAFBQrQ35a4xjQxsbxqSjsWkIeee9996L4vXr10dx165d8/bWrVujPkvGUyxfvjyKe/Xqlbc7deq0S8e5u5B30Jg0hJzT1JFzsDvU2gVVYxwb2tgwJh2NTUPIO5deemkUT5kyJYrPOeecvL1mzZqor0WLOIU/+OCDZV/71FNP3anjCi/emjXbfYMZyDtoTBpCzmnqyDnYHRjyBwAAAAAFcUEFAAAAAAUxbToA7GbPPfdc3r7xxhujvtatW0fxsmXLoviSSy7J282bN4/62rZtG8VjxoyJ4vvuuy9vP/LII1HfddddF8VhrZa0e4f5AQDQmPAvJgAAAAAUxAUVAAAAABTEkD8A2EVvvvlmFF9//fVR/NZbb+XtAw88MOqbPn16FLdp0yaKu3fvnreXLl0a9Q0bNiyK02nTw1kA06GFl112WRQPHDgwii+66KK83bNnTwEAgNK4QwUAQAPWu7dkVv6vd++6PkIAaNy4oAIAoAFbtGjX+gEAu4YLKgAAAAAoiBoqAChhy5YtURxOWX7TTTdFfS+//HIUt2vXLooPPfTQvN2+ffuo78MPP4ziGTNmRHFYU5XWMqXH+Nprr0Xxv/zLv+TtLl26RH2rVq2K4gULFkTxV77ylbx98803R329evWK4q1bt0YxU64DAJoS/tUDAAAAgIK4oAIAAACAgrigAgAAAICCqKECgBLCmqnUlClTorh3Mi91+txwPah0rajTTz89it94440oDmubfvrTn0Z93/ve96L4xBNPLHscaa1W27Zto7hjx45RHNZF3XXXXVHf5ZdfHsXUTAEAmjL+FQQAAACAgrigAgAAAICCuKACAAAAgIKooQKAaghrn9J6pB49epTdVpI2b96ctzt06BD1LVmyJIqPOeaYKF60aFHevu+++6K+/v37R/F+++0XxWvXrs3bGzdujPo2bdoUxeF6V1JcFzZv3ryor9IaXQAANDXcoQIAAACAgrigAgAAAICCGPIHANXwzjvvlO1LhwBu2LAhisMhce3bt4/63n333ShetWpVFPfp0ydvp0P8Fi5cGMVz5syJ4nB4Ya9evaI+M4vidBjf6tWr83b6/lauXBnFXbt2FQAATRV3qAAAAACgIC6oAAAAAKAgLqgAAAAAoCBqqACgGt5///28ndYUpbVM4ZTjUlwXNX369KhvxYoVUbxgwYIoDqczT7edMGFCFHfv3j2Kw2nU33vvvagvrZlas2ZNFKfvITRjxowoPvzww8tuCwBAY8cdKgAAAAAoiAsqAAAAACiICyoAAAAAKKhJ11A55yrGzZrV3PXmCy+8kLePOuqoGnvdnbF27doobteuXZ0cB9AQhTVUrVu3jvrS79bmzZujuFu3bnl77ty5Ud/y5cujeI899ojicF89e/aM+vbff/8obtmyZdnXSuu+Bg8eHMVPP/10FIfrZaW1WdOmTYtiaqiApiE9T0prPvfcc8+8nebFn/3sZ1H81a9+NYrDc5JWrVpVPI60BjRc6w+oC9yhAgAAAICCuKACAAAAgIK4oAIAAACAgpp0DZWZVYwrueSSS6L43XffjeKPfexjUfzMM8/k7f79+0d9e++9d7X3m9ZmtGhR+T/hj3/847x9//33R31jx46t9n6Bpi6sQUrXbJo5c2YUr1+/Por79euXt8N6KmnHuqcPPvggisMaq3Xr1kV9q1evjuIBAwaUfe20xmDlypVR/NJLL0XxsGHD8vaJJ54Y9aXvF0DDldZFhedCs2fPjvouu+yyKL7ooouiePz48Xn70ksvjfruvffeKP7rX/8axXfddVfePvXUU6O+tFarbdu2UXzhhRfm7TTHpu8P2B24QwUAAAAABXFBBQAAAAAFNbohf1u3bo3iXRnWl97qPuSQQ/L22WefHfWNGjUqitPhNeEt6K997WtR38MPP1ztY6pqiN8f/vCHKL7nnnvydjpMacaMGdXeL9DUrVq1Km+n0wGn3610WG/Yv++++0Z96RTsr776ahQvWbIkbw8dOrTifjdt2hTF4dDDdIhMeoy33nprFP/nf/5n3k6HGqbvH0DDVem8KB1G/Mgjj1R8rQcffDBvn3DCCVFfutzChg0bojgsf3j++eejvnQ5iVRV50bA7sYdKgAAAAAoiAsqAAAAACiICyoAAAAAKKheDjqtNIVn2p/2NWtW+Rpx48aNeXvhwoVR38iRI6M4nR70yiuvzNsHHnhg1DdnzpwoTmsM9t9//7z99NNPR31dunSJ4m9961tR/OlPfzpvp1Ms//3vf4/iG2+8MYrD7UeMGBH19e3bVwCqJ/yOp3VPac3kF77whSi+7rrr8nb6HU5zVlirJcXTqC9evDjqmzRpUhSnealVq1Z5O11yIZ1yPZzaXYprrtJaLaYhBpqGdHmVWbNmRfE+++wTxbfffnveDs97pB3rxdu1axfF4flcOk36kUceWfE4Hn300bz9xS9+MerbsmWLgN2NO1QAAAAAUBAXVAAAAABQEBdUAAAAAFBQvayhqmqtqEr9L774YsXnXnPNNXk7rSFK12FJ17SaN29e3k7XikmF679Icc3BJz/5yaivU6dOUXzTTTdF8W233Za3O3ToEPUtXbo0itPxzIcddljefuWVV6K+tFYDQHnhmP7u3btHfStWrIji9Ps/aNCgvJ3WMqXrwYV1nlKcH9Jazfnz50fxEUccUfa5c+fOjfrSXJKuuxfWWKVrwKQ1Vek6VemaV0BTUq7GsFI9uLTjOUdam1lJmlfCNemqep2w1lKSrr322ryd5oU0F/Tu3TuK/+d//idvh2t3SjvmheOOOy6Ku3btmrfT+vBwPT5px/qsBx54IG+nNVSsUYXawB0qAAAAACiICyoAAAAAKIgLKgAAAAAoqEEOLJ05c2beTmsX7r777ihO6xO+/e1v5+10rah0Xaq0PxyjnI4jTtc5SMdCf/jhh3l7w4YNUd8//dM/RfHpp58exW+++WbeTtde2HvvvaP4+OOPj+KwhuLee++N+tJx0wC2S2uZwjhdOyqtDUjjsOYozVkf+chHojjtD9eeSmuX0rXzwjyTbp/uJ62hbN++fRSH9QxprWZaN5HmzgEDBghoqqqqA6/udpXWe0ufm9YJ7UzdULh2lBTXag4fPjzqS8+LunXrFsV9+vTJ22HduSRdfPHFUbxo0aIo3m+//fJ2ei7TsWPHKP7yl78cxWGevPPOO6O+tKYK2B24QwUAAAAABXFBBQAAAAAF1dqQvw0bNujtt9/O43vuuSdv9+zZM9o2HbYSTt8rxdOBhsNSJOnYY4+N4nTaznC683TYTnpLOZ1qNBzWt2zZsqgvHfKSHnM4jXI65C+dYjkdejNkyJC8feSRR0Z9Xbp0ieL0uB5++OG8nd6anzZtmgCUFg4tlqTWrVvn7TAHSdLKlSujOBz2IsXDb9Lpjdu0aVPxtT744IO8nea3t956K4rToXmhdBhimt/S4wqnVU+nWA+PSdox3wFNWaWhepXszDTpqfT7e/PNN+ftCRMmRH3psg/nn39+FIfTmd91111R3xtvvBHFaS48/PDDyx7jb37zmyi+/PLLozg8zvR8LF0SIl0iJoxff/31sscA7C7coQIAAACAgrigAgAAAICCuKACAAAAgIJqrYZq8eLFuummm/J40qRJeTusTSglnf4znAp8yZIlUV9af5DWZ7Vr1y5vv/POO1Hf1KlTozid8jOczjite0rrvtJp1EPp+01rKA4++OAofu211/L2DTfcEPWldWAHHHBAFIdTq6bbDhw4sOwxAk1dOn15pRqqAw88MIrTacXDXJLWSKZToaf7Db/D6euGdamljius5UinSU9rLnr06BHFYb6oqs4zzbtAU1bdadNT6XlDWFMV1kNLO56fpHVSYV4577zzor7nn38+ivfff/8onj17dt5Oz7HSc530HKuS9HMJpzqX4ve/bt26qC+d2v3EE0+M4jAnpfVV7777brWPESiKO1QAAAAAUBAXVAAAAABQEBdUAAAAAFBQrdVQdenSRWeeeWYeh+tHvffee9G2y5cvj+J0jZP58+fn7bCeSpLmzJkTxWl/WDe1du3aqC+t1UprjsLXStd0GT58eBSn68WE67Y8+OCDUd+TTz6p6ko/i3SccSqsGWvVqlXUl9ZuANguHd8f1hyldY9pbVNayxTWHfTq1SvqS9elS/NQuP3YsWOjvnRNmAEDBkRxuE5dWvuQvod0DZwwX6S1D+n7S2usAGQqrUm1devWKK60DtXEiROjOP0+t2zZMoqvuOKKvD1y5MioLzwvkKTp06dHcVhPmdZmpe/nzjvvjOKLLrpoh2MvJ80jc+fOzduDBw+O+tI61YceeiiKzznnnLx90EEHRX1Tpkyp9jEBRXGHCgAAAAAK4oIKAAAAAAriggoAAAAACqq1Gqo2bdpEayR95CMfydt9+vSp+Nx0bYZw7HC4XoK0Y93D3/72tyg+//zz83Y6Jrdbt4/5XqIAACAASURBVG5RnNYc1ZTTTjstih9//PEoHjFiRBSHtVzpGOt0PZh0fHNYJ7ZgwYKor6r6K6ApW7p0aRR36NAhb6dj//v37x/FaU1CWK+Y1kyl9VdpTWlYnxTWnko71kGltRFhf7ruVFXr/4XvMd02zTNpPQfQlIXfj0prUqb1kulacbNmzcrbYX2RtGPddlpPeeWVV+bt++67r+J+9t577ygOz4WeffbZqO+QQw6J4vScK6zzPO6441RJeq6zaNGivH3WWWdFfel508knnxzFZ599dt5Oa83JT6gN3KECAAAAgIK4oAIAAACAgmptyF/z5s2jacfDW87PPPNMtG06NCWdDrRz5855e9iwYVFfOoztq1/9ahSH0wpv3Lgx6kuH+KS3jUPpNMFpnA6JCW/t9+3bN+pLh9O8+OKLURzeFk+HGqVDANPhBeHnkU6Tng5xBLBd+p3eY489yvZ17949itNhMGHuS4fprlixIorTYUDhsN10eOCyZcuiOB3asnDhwrwd5k2pcn6T4jyc5uT0GNNcCjRl4TID6XelkvS84c9//nPefvPNN6O+9LueTqs+derUvB0u2yJJS5YsieJHHnkkii+77LK8/dxzz0V93/3ud6M4zDGS9P3vfz9vp0P+Vq5cGcU9e/ZUOenrpsJjTKVTvaflHcDuwB0qAAAAACiICyoAAAAAKIgLKgAAAAAoqNZqqFLhNJ3plJ2pmTNnRnFYv/D2229HfWk9QjjluBSPQ06nL+7YsWMUp7Vb4bjotA4inc44rXUKx0anY4N79OhRcb9bt24t+TqStHz5clUSTvWcHuO+++5b8bkAtgu/02lNURpPmzYtisM8lOakNGeFeUaSunTpUvIYpB1zRTqNelifmdZbpnVPaV4K601TaV0ISzAAmbVr1+qll17K45tvvjlvp3XM6fcozQ1hf/hvubRjnWZaIxkuk/Lyyy9HfelyMum5UCit20zroFJhvdZHP/rRqC+tPT3hhBOiOMx199xzT9R36aWXRvGgQYOieNSoUXk7nWL+l7/8ZcVjBmoCd6gAAAAAoCAuqAAAAACgIC6oAAAAAKCgOquh2hkDBw6s9rbDhw/fjUcCoKlIa5nCeqW0hnL69OlRfPjhh0fxfvvtl7fTWqW0tildIyaso0jXmUvjtMYqrHdI6y9btWoVxWGtZvpa6TGGa3JJO9aQAU1VmzZtonWPLrjggrydfrfTGuhK61um606l26bf0auvvjpvp9/ttF48XZMyXMcprc36xje+EcVpDXhYc5XWW/3gBz+I4nnz5kVxnz598naar8I+acd60Xbt2uXtMN9K5CfUDu5QAQAAAEBBXFABAAAAQEFcUAEAAABAQQ2ihgoAals6hj+sZUrrq9I13v7t3/4timfPnp23x48fH/WlNQhTpkyJ4jfeeKPsftIaqnTNmLDua/78+VHfueeeG8VjxoyJ4rD+IT2mVLp+DtBUNWvWLKrn+djHPlaHR1O/pOtfAY0J/woCAAAAQEFcUAEAAABAQQz5A4AS0mF9oXSo3ZFHHlnxtQYMGFCyXcrRRx9dti+d/njDhg1RnE4lvCvCoYiVPotSxwUAQFPCHSoAAAAAKIgLKgAAAAAoiAsqAAAAACiIGioAKKF169ZRXKmOKJyevJSw5qp58+ZRXzo9e6X9pNOT70rNVFX77dChQ95Ojzmtmdq4cWPh4wAAoKHjDhUAAAAAFMQFFQAAAAAUxAUVAAAAABREDRUAlLB06dIo3rRpU95Oa4patCieStPapZ2pqdoVaR1U+p7CGqp0vauwT6q6hgwAgMaMO1QAAAAAUBAXVAAAAABQEBdUAAAAAFAQNVQAUEK4dpQU1wlt3rw56uvTp0+N7XdnaqaqqrcK+9O+qmqowjWuwvoxacf3n9ZUAQDQlHCHCgAAAAAK4oIKAAAAAApiyB8AlNCsWfx70+rVq/P2ihUror50eGAqHF6XDq3bFVUND9yVKdfDqeArDX+UpHbt2hXeDwAADR13qAAAAACgIC6oAAAAAKAghvwBAFDLrrrqqrJ91113XS0eCQBgV3FBBQAlfOlLX4ricePG5e20hmr06NEVXyusR6ov0hqxVDgVfDotfPp+OnfuXHMHBgBAA8OQPwAAAAAoiAsqAAAAACiICyoAAAAAKKjWBvaPGzduqZnNra39oZCP1PUBADWptvLOOeecs7t3Ua/96Ec/2pWnk3fQaHCu0yCQc1Djau2CyjnXo7b2BQASeQdA7SLnAE0TQ/4AAAAAoKD6N5cvAACoM6yRBQA7hztUAAAAAFCQOefq+hiAisxsiaRKRb7dJS2txkvVxXZN5dg+Qu0AGotq5Byp4X9n6/M+q7MdOQeNSg3mnfr6na2rfVZ3u10713HO1cif5P5TctMkN1lyEyX30Zp6bf/6x0juLzX0WvtJ7iXJbZDcfyR9J0nuTcnNlNxVweP9JfeKf/xeybXyj39NclMl91jw2JGS+3mF/beR3POSay65ZpL7lX+NKZJ7TXL9a/izO19yN+zC838iueNq8phq9v3p9fq6XVM6tsbwJ7nekrtHcrMkN85/rwcXeJ3Okru4Qv/lPl9OldzdktvDP36c5Mb7x38vuRb+8c/67V+UXDf/2L6Su7fCPkxyYyXXcXe+N8n1kNzjdf3frvb/v9Lwv7P1dZ87s119+dtd50CSe05yBxfZRnJf9ecsTnLdg8fNn3fM9Mc7Kug7T3Jv+7/z/GOtJfe4z0vhd/+W8Lkl9v9pyX3Ht4f445wouemSu6WGPp8qzw3DbSR3quS+V9f/fyn+fhv2d7YxHFu5vxoZ8memwySdKmmUczpQ0vGS3quJ164JZjvUii2TdImknyTbNZf0G0knSxoq6fNmGuq7r5f0c+c0UNJySf/iH/+CpAMl/UPSJ8xkkr4t6fsVDunLkh50TlsknSVpT0kHOqfhks6QtKLI+9wd/Gfya0nlB9UDjYD/7j4k6TnntK9zGi3p/0nqVeDlOku6uMx++irLPwc7p2GSmkv6nJmaSfq9pM/5x+dKOs8/7WuSDpH0P5LO9o/9t6SrKxzDKZImOadVu/O9OaclkhaY6YgCrwU0ePX4HOj/lB1LetfjZEmD/N+Fkm6SJDN1lXSNpI9KOlTSNWbqIukTkv6u7FznHL/tCEnNndP4Cvv/pqQbfftXys6hDnJO+ys7r6gLf5V0mpna1tH+0UjVVA1VH0lLndMGSXJOS53TfEky0xwzfddM4800xUz7+cfbmek2M71qpglm+pR/vJ+ZXvTbjzfT4enOzHSIf86+ZhptpufNNM5MT5ipj9/mOTP9wkyvS7o0fL5zWuycXpO0KXnpQyXNdE6zndNGSfdI+pQ/GTlO0p/8dr+X9OlthyOppaS2/vW+KOlvzmlZhc/rC5L+HHx2C5zTVn9s85zTcv8e1pjpB2aaZKaXzbKTHzP1MNMDZnrN/x3hHz/UTC/5z+YfZhpS4rP7pN+mu5lO9O3xZrrfTO2D/2bXm2m8pH9yTnMldTNT7wrvCWjojpW0yTndvO0B5zTJOb1oJjPTj8001eexsyTJTO3N9EyQ3z7ln3qdpH3NNNFMPy6xrxaS2vgfe9pKmi+pm6SNzuktv81Tkj7r21sltfbbbjLTxyQtdE5vV3g/YZ7Z3e/tYb8/oCmqdA70Hf/v9FQz3eLPJ7ado1zvz4He8t9pmamNme4x03QzPSSpzbadmOkmM71upmlm+m5VB+WcJjinOSW6PiXpDv/D+suSOvtzp09Ieso5LfPnIU9JOknZuU1bZec65l/j+8p+PC7JTIMlbXAuH0LVR9K84Nim+O1KnvOZ6Rj/Gf3JTDPM9MfgszvJPzZe0meCfVZ5DuScnKTnlF0AAzWnZm5Buvb+Nu5bkrtRckcHfXMk9zXfvlhyv/PtH0rui77d2T+3neTaBsNfBknudd8+RnJ/kdzhfrjKPpJrKbl/SK6H3+Ysyd3m289J7sYqjvu/FAz5k9yZ247Px+dI7gbJdZfczODxvSU3NdhmguTulFwHP8SmZYV9tpLcwiDey39GEyX3U8mNDPqc5E7z7R9J7mrfvktyR/r2PpKb7tsdtX2I0PGSe8C3z/fv4wxlQ4a6+Pf0guTa+W2uDG7Nz5HcN5Pj/q3kPlsT/3+p6T9JF9bX7ZrSsTX0P8ldojJDdZUNuXtK2TDdXpJ7V3J9JNdC24fUdfdDaExy/bbliDKvd6nk1khuieT+6B8zyc2VH7ojuV9Kbopvn+Dz3qOS6yS5JyXXtYr3M1dyHWrjvUmu77ZjbSp/jeE7W1/3uTPb1Ye/Ks6BugbtPwT/pj8nuZ/69imSe9q3vx6cxxwouc1BTujq/7e5f/6BwWuVHRbo/00Ph/z9Zds5hI+fkdzBkvuPbecZ/vFv+8da+POOCZI7W3KnS+6/qvhMvrTt/QXxSsn9TdmQ587+8UrnfCuVnSM1U1amcaTk9pDce35bk9x92j6cr9w50DEKhgVK7guS+3Vd//+m2P/XGvZ3tjEcW7m/Gpk23TmtMdNoSR9T9kvovWa6yjnd7jd50P/vOG3/NeFESaeb6T98vIekfZT9UnuDmQ6StEXS4GBX+0u6RdKJzmm+mYZJGibpKct+M2kuaUGw/b018f4qcU5/kPQHKfslStlt7ZPNdK6yW/7fcP7uk9ddwZA+5zTP/4pynP97xkz/5JyekbRR0l/8puMkneDbx0saapa/Zkd/d6mTpN+baZAkp+zXpG2Ok3Swss9ulZlOVTas8f/867SS9FKwffrZLVY2NLHecc7dUl+3a0rH1sgdKelulw3TXWSm55UNwfubpB+a6Shld5H6qophdJYNofmUpP7KcsH9Zvqic7rTTJ+T9HMztZb0pLIcKOf0lLJfi+Vzy2OSBvv8uVzSpc5pXbKrrs5pdS29t3qbH3aXxvCdra/73Jnt6oMqzoGONdM3ld3h6SppmqRH/VPDc6N+vn2UsvMIOafJZpoc7OqfzXShsjvcfZT9Gx727xbOabP8UGMztZT0hLLROz9Tdt52h3N6JHlaH0lLgtf4XzM9oeyO16ckfcWyYYMtVf6c71XnsrtaZpqo7DNaI+kd5+/Om+lOZcMWpcrnQKEGm68a+ne2MRxbOTW2DpX/x/g5Sc+ZaYqysf+3++4N/n+3BPs0SZ91Tm+Gr2Om/5K0SNIIZUMSPwy6Fyi78Bqp7MLLJE1zToeVOay1O/k23pe0dxDv5R/7QNkt8RY+sWx7PDzuPSUd6py+509IjlNW3/Bx+RMhb71/DzmXDRP4m6S/mWmRsuGEzygbouP8ZuFn10zSGOeiz0ZmukHSs87pDDP1U/bfY5tZkgYoS1avK/vsnnJOny/zWaSf3R7+2IHGapqkM3fyOV+Q1EPSaOe0yUxzlHy/Szhe2QnBEkky04OSDpd0p3N6ScqH/pyo+ORClo37P1/Z0Jy/KPuB6kx/HL9N9rPZTM38Dzq7+72RH9CklToHMtM9ymqIDnZO7/nzm/A7VOrcqCQz9Zf0H5IOcU7LzXS7qs415ZQ713lf0jHJ488lz71Y0h2SxkhaqawOfKy0wwXVemUXODmXDYO8TdJtZpqq7Afx01T+nG9D0K7yM1I2DLHcOVCIfIUaV1OTUgzxvwhsc5CqnvrxCUlfC8bEjvSPd9L2mqJzlN112maFpE9KutZMx0h6U1IPywpCZaaWZjpgF97Ka5IGmam/mVpJ+pykR/xFzbPafkJynrbXJmzzfUnf8e02yn4d2SrFhY8uG5fc3CxLhGYa5S/GZFlR+oGq+rN7UlmRuvzzDvLNTtp+oXd+8py5yuox7vCf0cuSjjDTQP8a7fyY53IGS5paxXEBDdlYSa39L8CSJDMd6GsbXpR0lpmam6mHsl+RX1X2nVvsLziOlfQR/9TVkjqU2c+7ksaYqa3Pfx+XNN3vr6f/39aSrpS21zx5V0j6lXPapAp5xntT2Y8otfHeyA9osiqcA2274FnqR5FU50eNF7T9btAwZecEktRR2Q+dKy2rpz55Fw75EUnnWlY/OUbSSue0QNl52Ylm6uLvpJ/oH5M/ni7Kao/uUJZztirLQW3SHSjLaQOD557k727JsnrsbsrOVyqd85UyQ1I/M+3r4/BH4UrnQCHyFWpcTU1K0V7ZbdY3/O3poZL+q4rnfF/Z7djJZpqm7bPi3ajsl51JkvZTcqfEOS1S9oX+jbI7VWdKut5vP1HacRKLlJl6m2mepK9LutpM88zU0d99+qqyBDJd0n3OaZp/2pWSvm6mmcoSwa3B6430x7Zttpu7JE2RdISkx0scwpPKhtlIUk9Jj/pfayZL2izphirewiWSDjbTZDO9Ieki//iPlF1sTlCJX3Kc0wxlvzrfryw5ny/pbv/f7CVln/cOfBIcqOzOVr1iZieZ2ZtmNtPMSs5EaGa3mdliM6uYQM1sbzN71szeMLNpZnZpiW32MLNXzWyS36ZiYbCZNTezCWb2lwrbzDGzKWY20cxKfsZm1tnM/mRmM8xsupntcFfWzIb419j2t8rMLivzepf7459qZnebWclfOs3sUr/NtHKv1Vj4H07OkHS8mWb5vHStpIXKZsibLGmSsouTbzqnhZL+qOy7OEXSucr+sZdz+kDZcNqplkxK4ZxeUTbBzXhleaKZsqHMknSFmab7fT3qnMZue15wF/xh/9Cvlf0IdJGynJP6q/yvzbXw3o71+2sSairvVCfn+O2qnXdqKuf47Wos7zTynFPyHMg5rVB253iqsvOK16rxWjdJau/zwPeUDQeUc5okaYKy7+Fdymbwq8hMl/hznb2UnWv9znc9Jmm2pJn++C72+1im7FzsNf/3PRdPsPUdST/wFz9PKLubPkW+7CHxgqSR2340V3ZxNtWfqz0h6QqfZyqe86X8yJwLJf3VskkpFgfdFc+BAg0uXzWVcx2/XcW8U2/PdXalAIu/Yn+SGyW5P9T1cezE8Z4hue/X9XHseFxqru1DGVspOyEcWmK7oySNklR2kgC/XR9Jo3y7g6S30tdTNlSyvW+3lPSKpDEVXvPryv7xK7tOhqQ5krpXcWy/l3SBb7eS1Lkan81CZYvQpX19Jb0jqY2P75N0fonthik7EWir7B+npyUNrOv/7vxV70/ZxBJP1dK+XpBcl7p+z7XzXmsu71Qn5/i+auedmso5frsayTvknKb5p2xinePr+jiSY+oluWfq+jh27pibzrmO367aeac+nevU1B0q7ASX3cl61qzKW9v1RQtJP63rgyjBT3PvZjvn8mnu042ccy9IFaex37bdAufceN9erewuZd9kG+ecW+PDlv7PqQQz20vZENXfleqvLjPrpCxR3uqPYaNzrqq1yj4uaZZzrtzwUT9tt4XTdqf2l/SKc26dc26zpOcVTFGL+s1lQ3h+a6aOu3M/fpjgz5xf7qEJqLG8U52c4/uqlXdqKuf416rpvEPOaXp+qNLDkevSPpK+UdcHsZOaxLmOf62dzTv15lyHC6o64pxuc1kRa73nnO53rv4sNhzoq3jxxHkqcTJShJn1Uzak9JUSfc3NbKKyoQZPOed22Mb7hbKFDbeW6d/GSXrSzMaZ2YUl+vsrmy3pf/0t9d+ZWbsqXvNz/7+9ew+3qyrvPf57c9+5kStJCCThGi4hBEGstYBIKfSilCOnlUoVfFrrKajQHlu8CwdBq3Jq6cFaBGsrIhRFQYKCCiWixBDIFRJIMEAukAQaQu4X3vPHHHsxx9hrzbX2ZO291977+3me/TDHGmPNOeaC9TLHmuMdU9KtVQ/mvk7ZQ62fU7bQyyvufl+VpssknWpm481suLIHxR5SpR1alLtud9fWLj7GJn99GmJ/0CVxpyjmhPpG4k6zYo7UxLhDzOmf3PWid1z9r0e5a4G7FvV0Pzqpv1zrSJ2POy1zrcOACkiY2UhJ35N0mbt3uBh19/3uPkfZvPRTzGxWlX38kaSN7r6wgUP+jru/SVmS8SVmdlpSP0jZbfyvufuJyuaYV51DHY49RNK7lOXKVavPL9t9kKQRZnZh2s7dn5T0RWU5fz9WlqPYK34EAHqTejFHqh93mhxzpCbGHWIO0Hpa8FpH6kTcabVrHQZU6M1qLf1ampkNVhZgbnH37xe1DbehH1D2XI3U2yS9y8zWKLs9/w4z+3aN/awL/9yobIGAU5ImayWtzf06dIeygFPL70t6zN1frFEflu32Te6+V6os212tbze5+0nufpqy5x09VXBcoD9oatzpTMyRCuNOM2OO1Ny4Q8wByusv1zpS5+JOS13rMKBCbxaWubdDwy8V71HHZ2E0zMxM2bzdJ939uhptJprZmLDdpuxhyyvSdu7+cXc/2N1nhH793N07/DJiZiPMbFT7tsJKSMm+XpD0vJnNDC+dKemJglO5QDVugQdh2W4bHs65smx3lf6FZbxtmrI5xdVWkwP6k6bFnUZiTmhXN+40M+aE/TUz7hBzgPL6xbVO2F9n4k5LXes07cG+QHdz931m1r7M/UBJN7v78rSdmd2qbPnoCWa2VtJn3f2mtJ2yX1r+XNLSMG9Ykj7h7nNzbaZI+paZDVT2g8Tt7l5zmdAGTJJ0Z/Zd1yBJ33H3akvtf1jSLSGYPiPp4mo7C4HqLEl/VeuA7j7fzNqX7d6nbCneWk8I/56ZjZe0V9IlDSSlA31ak+NOIzFHam7caTTmSE2KO8QcoLx+dq0jNRB3WvFax9yrLtoBAAAAAKiDKX8AAAAAUBIDKgAAAAAoiQEVAAAAAJTEgAoAAAAASmJABQAAAAAlMaACAAAAgJIYUAEAAABASQyoAAAAAKAkBlQAAAAAUBIDKgAAAAAoiQEVAAAAAJQ0qKc7ANQzYcIEnzFjRk93AwUWLly42d0n9nQ/gGZolZizeLG0b1/t+kGDpBNO6L7+tBJiDvqaVok7qK0o7jCgQsubMWOGHn300Z7uBgqY2bM93QegWVol5pgV1+/bJ7VAN3sEMQd9TavEHdRWFHeY8gcAAAAAJTGgAgAAAICSum3KH3NDWx9z0tHX9Ia4s3v37qg8dOjQpu17586dle22tram7beZiDvoS3pDzElt3rw5Ku8rSNwbMCD+HX7IkCFRecyYMc3rWBch5qArdNuAirmhrY856ehrWjHu7N+/PyqvWbMmKh9++OGl9zVw4MCovHTp0sr2rFmzojqrl6DTTYg76EtaMebUc+ONN0blLVu2VLbTwdXIkSOj8sEHHxyVzzvvvCb3rvmIOegKTPkDAAAAgJIYUAEAAABASSybDgDdaO/evVH5+eefj8pFU/7cPSqnU/xS69evr2wff/zxjXYRQA9Iv99F03LTtunUvMGDB1e206nBgwbFl35p3mbRcdO6fJ6mJJ1zzjmV7XvvvbfmfqSOfU77BfQm3KECAAAAgJIYUAEAAABASdxfBYBuNGzYsKj8jW98Iyqnyw7PmTOnsl1vZb4f/vCHUfmrX/1qZfvss8/uVD8BdK+iKX+vvfZaVJcuX56f4pe69NJLo3I6xW/KlClROb8U+q5du6K6PXv2ROVRo0ZF5UWLFtXsRyqd4pefmlhvOjPQarhDBQAAAAAlMaACAAAAgJIYUAEAAABASeRQAUA3SpdNnzdvXlResGBBVJ49e3Zl++KLL47qrrrqqqic5jvMmjWrdD8BdK80LyofK4pypCRp7ty5UfnLX/5yZXv16tVR3bhx46Jymps5derUynb+0QtSxyXY0/fm88DS3KyPfexjUfmyyy6LyuRNoTfjDhUAAAAAlMSACgAAAABKYkAFAAAAACWRQwUA3SjNhZg8eXJU3rdvX1ResWJFZfuSSy6J6tJnWo0dOzYqT5w4sXQ/AXSv9FlTRXlTF1xwQVS+/fbbo/LIkSMr28OHD4/q0rynbdu2ReUNGzbUPO7OnTujcltbW1TO51jt3r07qvvkJz8Zlb/0pS9F5euvv76yff7550d1aVxMn2EF9DTuUAEAAABASQyoAAAAAKAkBlQAAAAAUBKTUAGgB6U5COvWrYvKo0aNqmyPGTMmqhs6dGhUTp9DNWLEiGZ0EUAPe+CBB6LyD37wg6g8ffr0qJx/hlWaf5Tas2dPVF6zZk1l+9hjj43q0ryoLVu2ROV8Xmea45nGo/SZfB/4wAcq23PmzInqjjjiiKicf96V1DEvDOhu3KECAAAAgJIYUAEAAABo2OTJkln1v2Tx2n6BKX8A0IPSKTWrV6+OykVLJ6d16ZS/qVOn1nwvU2aA1jJgQO3fuL/+9a9H5YEDB0bldFpffvny9Lteb3n2fHn9+vVRXTrNuCiOpHVpH9Pj5s//8ssvj+ruvvvumsdBz3jxxXJ1fRV3qAAAAACgJAZUQE8oulfeX++XAwAA9EIMqICeUO9+eH+8Xw4AANALkUMFAF0sn0uQzv1PlxIeNCgOy0XvnTRpUlR+6aWXar4XQO+S//7+4he/iOqGDx8eldMlyItymdK2aV5UPj8rzbfavn17VE4f+5A/Vr34k+ZUjR49urL90EMPRXVLly6Nyscff3zhvoHuxh0qAAAAACiJARUAAAAAlMSACgAAAABKIocKALpY0TNTVq1aFZWLnkWze/fuqPzqq69G5fHjx0flZ599tlSfAPS82267rbL98ssvR3X5fCOpY65T/vt9wAEHRHU7duyIymlOVf4ZVmmOZ3qcNCYNGzasah+k+jlVRflXX/nKV6Lyv/3bvxXuC+hu3KECAAAAgJIYUAEAAABASQyoAAAAAKAkcqhybrjhhqi8bNmywvoi6fxf8hUAVPPAAw9E5WnTpkXlwYMHV7bT/IVUGmdWrFjxBnsHoKf88pe/rGznnw0ldcx7Sg0ZMqSyvXPnzsL35mOMFD8fasyYMYXHSa918vlXaT5oveui/HHT8503b15hP4Cexh0qAAAAACiJARUAAAAAlNRjU/7yt6Db2toabivFt7LrSW8b5/3oRz+KyuvXr4/KBx54VMeGNQAAHgdJREFUYFR+3/veV9n+/Oc/H9UdcsghUbloil/+lng1RX0G0Ls9/fTTUXnixIlReejQoTXfmy5/nMaZtLxhw4YyXQTQAh577LHKdr3pc+l1UT4W7Nq1K6rLL20uxVPt0vemMSSNMUXXY3v27Clsmx43f05pHBw+fHjN4wCtgDtUAAAAAFASAyoAAAAAKIkBFQAAAACU1GM5VPl8pEsvvTSqO/3006NyvRyrstJl0E855ZSonM73Pfjggyvbt912W1SX5ludd955UXnUqFGV7TRHKs2pSudGdwbLswOtLZ8XIXXMM0i/w/kljtPljdPciDTPYu3ataX7CaBnrV69urKdXjek1wnpIxXysWDQoPhSryh3KW2fxpR0yfV0X7X6UK+tFF8LpX3etm1b4XuBnsYdKgAAAAAoiQEVAAAAAJTEgAoAAAAASuq2HKrXXntN27dvr5Tzc/vvuuuuqO2OHTui8qxZs6LyuHHjKtvpswnSecTPPfdcVP7mN79Z2Z48eXJUN2HChKh89913R+Vzzz23sr1ly5aobu7cuVF5xYoVUfmwww6rbJ911llR3fTp01VWmn9VNI+a51sBPW/+/PlROc0zKMqprPcsmjT/asqUKZXtVatWRXVHHHFEgz0G0BNefPHFynZ6ffJGcpnqPb8uv6/0miJtm+473z7N+Uz73Jmc7zVr1kTlrVu3RuXRo0c3vC+gK3CHCgAAAABKYkAFAAAAACUxoAIAAACAkroth2rnzp1atmxZ1bp8bpUk3XLLLVF59uzZUTn/fKj0WVFpnsDSpUujcv6ZL6eeempUlz4f5uyzz47K+Xyt9LjnnHNOVN64cWNUfuqppyrbv/rVr6K6Y445Jiofd9xxUfnkk0+ubE+cODGqS/OiyJMCWtvy5cujcpqDkMaW/PNXinIdqtXncxZeeumlqI4cKqC15XMk0/+313t+XT4Xs17OVCqfB5XmbqU57mk53880/ypVLwe8yMqVK6Pym9/85obfC3QF7lABAAAAQEkMqAAAAACgpG6b8rd///5oqfGXX3759U4MirvxyiuvROU777wzKo8dO7aynS7DOWrUqKj81re+NSofddRRle10qk26PPvmzZujcv7Wdn7pdik+H6njcu7Tpk2rui11XP5z3rx5UXnBggU19ztmzJionC7BfuCBB1a2jz766Khu6NChAtC90uV/0yl+6TS+fDmNlem0n1T+vU8//XRU95a3vKVuXwF0n3Xr1tWsS6fppY9MaKb8vtNpeGl8Sq/B0uuqIul787Gw3vn95je/icpM+UNP4w4VAAAAAJTEgAoAAAAASmJABQAAAAAldVsO1YABAzRixIhKOb+M+MUXXxy1nTFjRlRO85N27dpV2U5ziIYNG1azrSQtWbKkZh9HjhwZldN8pXy+wgsvvBDVpXkQo0ePrvneNGcqXZY0zc/KS88nXZ59/fr1UTl/DldffXVUd+GFF9Y8DoCu8dxzz0XlmTNnRuU0ryAvzaNIc6rSfId8PkP6CAkArSVdCrxI0Xf9jcovfZ4+biFdvj295sr3q14f03ys/HVUvSXUN2zYUFgPdDfuUAEAAABASQyoAAAAAKAkBlQAAAAAUFK35VBt2bJFd911V6U8ZcqUynaa95PmGB122GFROf8cp3QObrqv3bt3R+X9+/cX9jEvfR7W4MGDK9v55ztJ9XOo8tLcrEmTJhX2MZ9/lc5XTsvpZ5f/PNL8i+uuu65mHwE0T/47neZBpnkFRc+WSvMK0u90Gu/y+Q5p3ieA1vLMM8803DbNn0yf25SPDWmMKWqbSp9XmV6fpDEpv+90v2k/0nK+fb0cqk2bNhXWA92NO1QAAAAAUBIDKgAAAAAoiQEVAAAAAJTUbTlUu3fv1qpVqyrlww8/vLI9a9asqO2yZcui8tq1a6NyPi8ozUeqN+82X5/mKqTldP5vfi5xOn83nWfc1tYWlfP5V6nNmzfX7KMkvfrqq5XtNM8rXyd1fJZWPl/j6aefjurSfQHoGs8++2zNujSGbd++PSrnY0dRvkK1cj7HMn3+FYDWkj5Xskh6fZLmNqXPi+qMfBypF3PSfuTLaZ/S66Q0hyr/DL6iayap4/NJgZ7GHSoAAAAAKIkBFQAAAACU1G1T/gYMGBBNbXnkkUcq2+lUu3Qp8LR+x44dle10efIJEyZE5W3btkXlomXT01vk6bKk+XJ6qzpdNj2Vv32dTstLb/Pnz0+Kl0JPl1zO3yKv1uf8svLpe6+88sqo/P73v79q3wG8MStWrKhZVzTtRYpjS9o2jWfpdJx8PFi3bl1jnQXQI1avXl2zLv3up9crO3fujMr1pswVyU/zO+igg6K6l156KSqn1xX5KX/p9Uh6LTd27Nia+077n+6LZdPRarhDBQAAAAAlMaACAAAAgJIYUAEAAABASd2WQzVt2jRdf/31UbnduHHjorbpMuLpvNt8TkGab5QupTlq1KionM8pSuckp3N00+VC83OU06VC0xyqtM/5Y6XHqdeP/OczZsyYqC7NN0s/y5kzZ1a2zzrrLBUhhwroGp3JX8rHqFS9JYzT/Kt8nEofsQCgtaTXPvlrgfS7ncaC9LohjQ1FdWk5f02yYcOGwuOmiq51Xnnllah8xhlnROV77rmnsp3GwTSnKs3lAnoad6gAAAAAoCQGVAAAAABQEgMqAAAAACip23KoBg4cGD1z4JprrumuQwNAj8rnL6V5BfVyI/K5BGldmsuZyuczFOVmAeh5aZ5jPm8ozRefPn16VE7zxefPn1/Znjp1alS3e/fuqFwUR+rFmFQ+RqX54elzQVP5a8Q0RyqNm0XPFAV6AneoAAAAAKAkBlQAAAAAUBIDKgAAAAAoqdtyqACgv8o/hyp9nkqaF5XmBhTlMKQ5Cmk5v+80byLN3Ur7BaB7pTlUbW1tle30GZtz5syJymmO0SOPPFLZTp8zVS8vKt++Xu5luq98Oa1L+5HPmZKko446qrL905/+NKqbMGFCVK73PCygu3GHCgAAAABKYkAFAAAAACUx5Q8AutjWrVsr20OHDo3q0mkwqYEDB9Zsm06pqTcFMC+dQjRp0qTCfgDoWul036JpuGeccUZUXr58ec22RXGgmnxcSZdjT5dvfyOPYxg/fnxUzk/rS6f8pedQL24C3Y07VAAAAABQEgMqAAAAACiJARUAAAAAlEQOFQB0sW3btlW2O7s8eT5XIM0byOdX1dt3ukz6li1bojI5VEDPSvMr05yqvHPPPTcqL1q0qGbb9LufLjle9LiFNObs2bOn8L359umjGlJDhgyJyqeddlpl+9prr43q0nzR0aNHF+4b6G7coQIAAACAkhhQAQAAAEBJDKgAAAAAoCRyqACgi+3atauyPWLEiKguzZNIy/l8hvSZL2nORZpTlc93OPTQQ2v2CUDPS3OK8kaOHBmV889skqTt27dH5XzOUZozlZaLvPrqq1E5zZlK41X+uGneUyrNg8rHszTWpX0uyi8DegIDKgAAAKAPu+KKKwrrv/CFL3RTT/ompvwBAAAAQEkMqIA+YPJkyaz23+TJPd1DAACAvokpf0Af8OKLb6weXevhhx+ubI8aNaqwbVtbW81ymmORPncqzVnIPxMmzZlauXJlVD7hhBMK+wWga6X5lfnn19XLeUxjQT4fKc17Sstp7mVR/lUaY9Jyft+DBsWXmMOGDYvKW7duLSznpfmi48ePr9kW6AncoQIAAACAkhhQAQAAAEBJTPkDgC72oQ99qLJ97bXXRnX5pc2ljssUb9iwobI9bty4qG7v3r1ROZ0SmJ9euGPHjqhu7Nix9boNoBvNnTs3Km/evLmyvXPnzsL3rlq1quHj1HtUQ36qcDptL53il04XzC93nt9PNUuWLInKn/70pxt+L9BqGFAB/QRLpgIAADQfU/4AAAAAoCQGVAAAAABQElP+AKCLXXXVVZXt448/Pqp74oknonKaK3HUUUdVtufMmRPVpXlRw4cPj8r5pdEvuOCCTvQYQE+bMGFCw23T/Mn8EuXpkuppOc3FzOcvpUudF+VbpdK26SMjjj766JrvBXob7lABAAAAQEkMqAAAAACgJAZUAAAAAFBSt+VQLVy4cLOZPdtdx0Mp03u6A0AzEXde92d/9mc93YVaiDvoM1ox5mzZsqWnu1DV1VdfXVjuQsQcNF23DajcfWJ3HQsAJOIOgO5FzAE6ry88J5MpfwAAAABQEsumAwAAAOgTiu54ddXdLu5QAQAAAEBJVvRQNqAVmNkmSUVJvhMkbW5gVz3Rrr/0bTq5A+grGog5Uu//zrbyMRtpR8xBn9LEuNOq39meOmaj7d7YtY67d+2f9EmXlru0xKVFLr2lSft90KWTS7WR5oW+LHJpvUs/CK+/3aVXcnWfCa9PdOkXLi1z6Y9z+/mhSwcVHP8yl94Xtn/Lpflhv0+69Lku+KzXuDSh5HsnuvTjLv/voQv+JD3aqu36U996w5/k+yVfJPlyyRdL/reSD+imY//PcNzXJD85qfu45KskXyn52bnXzwmvrZL8itzrt0i+RPJrcq99SvI/Ljj+iZLfFLYvknyT5I9L/rTkP5H8t7vw3CdK3ivjS+1z6v3f2VY9Zmfa9YU/ySdL/l3JV0u+UPK5kh9VYj9jJP/rGnUzQ+xr/9sq+WWhbo7kj4TXH5X8lPD6u0PMmif5+PDa4ZLfVtAHk/znko/uynPrizGlsc+hd39n+0Lfav117ZQ/s7dK+iNJb5L7bEm/K+n5Lj1mI9xPlfscuc+R9CtJ38/VzqvUuV8VXrtA0r9IOkXSZZIks3dKelzu66sew2yQpA9I+k545VuSPhiOOUvS7c09qTfAbJDcN0naILO39XR3gC60011z3HWcpLMk/b6kz6aNzLokv3SZpP8h6aHkWMdKeo+k4ySdI+kGMw0000BJ/y/08VhJF5jpWDPNDucxW9KbzXSAmaZIeou7flBw/E9I+qdc+TZ3neiuIyV9QdL3zXRM+qZmfBbu2iRpg5mIL0COmUzSnZIedNfh7jpJ0sclTSqxuzGS/rpahbtWhtg3R9JJknaE40rSP0i6MtR9JpQl6cOS3izp65Lan7twtaRPFfThDyQtdtfWrjw3YgpaTVfnUE2RtFnuuyVJ7psrAxCzz8hsgcyWyexfZWbh9Qdl9kWZ/VpmT8ns1PB6m8y+K7MnZXanpLbKUcy+JrNHZbZcZlc23Duz0ZLeIRVehEjSXknDJQ2VtD8Mli7T60GnmndIekzu+0L5QEkbwuewX+5PhD58TmY3h/N+RmYfyfXvwvA5LJLZ12U2sKHzzT6re2X2lzIbEfb/a5k9LrNzQ5uLZHaXzH4u6WfhnT+Q9N46nwXQJ7hro6QPSrrUTGami8x0l5l+LulnZhphppvN9GszPW6mcyXJTMeF1xaZaYmZjgxt7zHTYjMtM9OfVjnek+5aWaUr50r6rrt2u+s3klYp+/HmFEmr3PWMu/ZI+m5ou1dSm5kGSBosab+kq1RlYNjOTKMkzXbX4hqfxQOS/jV8HjLTg2b6RzM9KumjZjrJTP9lpoVm+kkYwMlMHzHTE+Fz+G547fTw2SwKn9uocBjiC9DRGZL2uutf2l9w12J3zQtx6UshpixtjytmGmmmn5npsfD6ueGtX5B0ePjufangmGdKWu1emV7mkkaH7QMktf9Q/Jqy657hkvaa6VRJL7jr6YJ9v1fSD7vp3IgpaB1dentSGhmmuD3l0g0unZ6rG5fb/g+X3hm2H3TpK2H7D1z6adj+G5duDtuzXdpXmc7Xvi9pYHj/7Ny+ak8LlN7n0h258ttdesmlxS7d69Jx4fUDXLrHpUddOtOlj7h0UZ1zv9KlD+fKn3Hpv12606W/cmlYeP1zLv3SpaEuTQjHH+zSMS7d7dLg0O6G3PTBWue7xqUZLv001/Yaly4M22PCv4sRLl3k0trk38NUl5Z26X8TXfAn6YOt2q4/9a03/Em+rcprWySfFKbBrZV8XHj9GskvDNtjJH9K8hGSXy/5e8PrQyRvC1Njbszt84CCPjyYn/In+T+3HyeUb5L8/PD3jdzrfy75P4ftfwzTc/42TNe5qc55nyH593Lli9r3lXvtjyW/N9fHG8L2YMl/KfnEUP5TyW8O2+slH9r+GYV/3i3528L2SMkHhe2pkve6+FL7M+3939lWPWZn2vX2P8k/Ivn/rVH3bsnvl3xgiFHPST5F8kG5KXUTlE0JNslnSL6sgWPeLPmlufIxYd/PS75O8unh9bPCNL27JT9A8vva42PBvp+VfFR3nFtfiymN/ffSu7+zfaFvtf669g6V+zZlt5Y/KGmTpNtkdlGoPUNm82W2VNndnONy72yfgrdQ0oywfZqkb4f9LpG0JNf+T2T2mKTHw36ObbCHF0i6NVd+TNJ0uZ8g6Xq137lyf0Xufyj3k0Obd0q6Q2Y3yuyOMLUxNSWcs8I+rpJ0sqT7lN06/3Gu7T1y3y33zZI2Krsdfqayz26BzBaF8mENnO8PJX1T7v8eyr8n6YqwjwclDZM0LdTdL/eXc+/dKOmgGp9Vy3L3f23Vdv2pb33E/e5q/078nqQrzJR+d34l6RNm+ntJ0921U9JSSWeZ6YtmOtVdr3RlJ911mWfTd74i6f9I+rSZPmmm2830l1XeEsej6iwp3xb+OVPZNOX7w2fxKUkHh7olkm4x04WS2u/GPyzpOjN9RNIY98rrvTK+1NIXvrOteszOtOvjfkfSre7a764XJf2Xsil4JukaMy2R9FNJU9XgNDozDZH0Lkn/mXv5f0m63F2HSLpc0k2S5K773XWSu96p7O74XElHmekOM91opuFVDjHOXa9207n1qZjSiN7+ne0Lfaul65dNz6a3PSj3z0q6VNK7ZTZM0g2Szpf78ZJuVHax0m53+Od+1XtWltmhkv63pDOV5Wndk+yr1vsmKJtSc0+ur1vDIFBynytpcGiX92lJn1c2GPuFpPdL+lyVI+zs0A/31XL/mrLB0QkyGx9qdudatZ+zSfqWXs/nmin3zzVwvg9LOqcyhTLbz7tz+5km9ydD3fakz8NCv4F+wUyHKfvObQwv5b8TJundYeAyx13TPJu29x1lFyQ7Jc010zvc9ZSkNykbWF1tps90ohvrJB2SKx8cXqv1er7/5yr74WmkpMPd9SeSzq9yodMxHnV0oqQnc+X2z8IkLc99Dse76/dC3R8qy/N6k6QFZhrkri9I+gtl07IfNtPRoS3xBehoubIfTzvjvZImSjrJs7ynF9XIdU/m9yU9FgYx7d6v13/I/k9l10YVIZ5cpOy7fmVo/wtVn263L0xHlrr+3IgpaBldvSjFTJkdmXtljrIlIdu/HJtlNlLS+Q3s7SG1J0WazZI0O7w+Wtn/+F+R2SRlwaIR50v6kdx35fo7OZfLdYqyz+elXP2Rkg6W+4PK5hS/JsmVz+d63ZOSjsi99w9zg5wjlV3EbSno388knS+zA8P7x8lsegPn+xlJ/60s8EnSTyR9OHdeJxYc8yhlifNAn2emicoWm/lnd3mVJj+R9OGQWC0znRj+eZikZ9z1T8ruCM8200GSdrjr25K+pGyA0ai7JL3HTEPNdKiy+PBrSQskHWmmQ8Ovyu8Jbdv7P1iv53K2SZVzGChpSHKMOB4lzHS6spkEN1apXilpopne2n7ckEc2QNIhnuVf/b2y3IuRZjrcXUvd9cVwDu0DKuIL0NHPJQ01y/IXJclMs0O+0jxJfxoWqZmobKbOr5V91za6a6+ZzpA0Pbz1VamSs1hLOjNHynKmTg/b75A65Eh9TNI/uWf5m8pizWtS1TtUK/X6bJquPjdiClpGV9+hGinpWzJ7QmZLlE1N+5zctyj7H/cyZRctCxrY19ckjZTZk8oSsBdKktwXK5v6tkLZinoPN9i396hjUDlf0jKZLVa2GtZ72idWBp+X9Mmwfauy2+QLJH21yv7vVRYg2v25pJVh6t1/SHqv3PfX7F22aMWnJN0XPrv7JU1p8Hw/KqlNZv+gbDrQYElLZLY8lGs5Q/k7dr2AmZ1jZivNbJWZVX00tpndbGYbzaww8JrZIWb2gJk9YWbLzeyjVdoMM7Nfm9ni0KZwERQzG2hmj5vZjwrarDGzpWa2yMwerdFmjJndYWYrzOxJqzLN1Mxmhn20/201s8tq7O/y0P9lZnarZXeNq7X7aGizvNa+epm2kNS8XNl0kvuU/eJaTeW7E9q3f3f+RNKyMP1tlqR/l3S8lC1UoWxxiKvTnZnpPDOtlfRWSfeY6SeS5K7lylb9fELZVOBLwjSYfcru6v9E2YDo9tC23SWSvuWuHcqm3g0301JJC93jH2vctULSAbkFIqTsYmaRmZ5StgLgu92jO1Tt792jLDZ+0UyLJS2S9NvKBm7fDsd8XNkF1xZJl4VE8yXKFtC4N+yq18WXWpoVdxqJOaFdw3GnWTEntGta3OnHMadQ+DHnPEm/a6bVIdZcK+kFZSvkLZG0WNng5O/c9YKkWySdHL5771N2PSB3vaTsrvCyaotSmGmEstVNv59U/aWkr4Tv9zVSNAA6SNIp/voKotcru+75kF5fxTjvHklv76Zz6zMxpZ7+cq0T2hXGnZa91nkjCVj81fnLFqA4ssf70Xh/H3JpbI/3o8E/ZRd0q5X9GjZEWWA+tkq705TdMShM1lWWZ/KmsD1K0lPp/pRNfxoZtgdLmi/ptwr2+TfK/qfzo4I2a1Tn+WHKlt3/i7A9RNKYBj6bF5Q9hC6tmyrpN5LaQvl2VVlkRdlgYZmyXyEHKRuAHNHT/975K/cn+eWS/0UPHv8hyXtNfKl9Hs2LO43EnFDXcNxpVswJ7ZoSd4g5/ecvLCxxfzcdq0/ElPrn2X+udUK7huNOK13rdH0OVf92hbL/cFuf2URJ18n9v3u6K50QlpX2Z9w9v6x0xN0fkvRy+nqVdhvc/bGw/aqyuwJTkzbu7Xl2WZAZLFWdLiYzO1hZjsk3Gj6j6vs5QFmgDInCvsezu7xFwrK4Xuup64MktVn2CIDhen2Z3LxjJM139x2eLf//X8qeo4Te6WuK8zW7TZjSc527elN8qaVpcaeRmBPqGoo7zYo5YV/NjjvEnH7AXRsk3WhWWYa9S/SxmFJPv7jWCfvqbNxpmWsdBlRdyX2lsv/AW5/7JrnXex5Xq5mq+EHRa1XlYqQMM5uhLEl/fpW6gZZN3dwo6X5379Am+EdJf6dsrnkRl3SfmS00sw9WqT9U2Qpt3wy31L9hZiPq7LPalNbsYO7rJH1Z0nPKno32irvfV6XpMkmnmtl4Mxuu7IGNh1Rph17AXbvc9R89dOxNXvzQ4d6kS+JOUcwJ9Y3EnWbFHKmJcYeY07+463Z3be3iY/SlmFJPf7nWkTofd1rmWocBFZCwbKGU70m6zN07/E/B3fe7+xxlq66dYtkiKek+/kjSRndf2MAhf8fd36RsgZFLzOy0pH6Qstv4X3P3E5UtSlJ1DnU4drVlcfP1Y5X9unWosiVnR5jZhWk7z1aD/KKyPKMfK8udqZ33B6CUejFHqh93mhxzpCbGHWIO0Hpa8FpH6kTcabVrHQZU6M3qLivdWWY2WFmAucXd08TdSLgN/YCkc6pUv03Su8xsjbLb8+8ws2/X2M+68M+NyhJ1T0marJW0Nvfr0B0qXkUuLIvrL9ao/11Jv3H3Te6+V1mC8m/X6NtN7n6Su5+mbPXIpwqOC/QHTY07nYk5UmHcaWbMkZobd4g5QHn95VpH6lzcaalrHQZU6M3CstJ2aPilIlpWurPMzJTN233S3a+r0WaimY0J223KVkxakbZz94+7+8HuPiP06+fu3uGXETMbYWaj2reVPUx2WbKvFyQ9b2Yzw0tnKlsRrpZqy+LmPSfpt8xseDjnM6WOq7uFPh0Y/jlN2Zziaqs6Af1J0+JOIzEntKsbd5oZc8L+mhl3iDlAef3iWifsrzNxp6WudYofmgu0MHffZ2bty0oPlHSzuy9P25nZrcqWcZ1gZmslfdbdb6qyy7cpW95+aZg3LEmf8Owhz+2mSPqWmQ1U9oPE7e5ec5nQBkySdGf2XdcgSd9x9x9XafdhSbeEYPqMpIur7SwEqrMk/VWtA7r7fDO7Q9JjkvYpW/K61hPCv2fZA6j3SrqkgaR0oE9rctxpJOZIzY07jcYcqUlxh5gDlNfPrnWkBuJOK17rmHvVRTsAAAAAAHUw5Q8AAAAASmJABQAAAAAlMaACAAAAgJIYUAEAAABASQyoAAAAAKAkBlQAAAAAUBIDKgAAAAAo6f8D81SK2orudDIAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 864x720 with 30 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "NgXr0mXFeOgb",
"colab_type": "text"
},
"source": [
"### Try Predicting with your own input images"
]
},
{
"cell_type": "code",
"metadata": {
"id": "nFR38XWJePEm",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 285
},
"outputId": "4ce7fc6d-247e-4011-8475-6710f5fc0aa9"
},
"source": [
"from skimage import color\n",
"from skimage.transform import resize\n",
"image = plt.imread('https://external-content.duckduckgo.com/iu/?u=http%3A%2F%2Fpngimg.com%2Fuploads%2Fdress_shirt%2Fdress_shirt_PNG8117.png&f=1&nofb=1')\n",
"image = color.rgb2gray(image)\n",
"plt.imshow(image, cmap='gray')\n",
"image = resize(image, (28, 28), anti_aliasing=True)\n",
"image = (np.expand_dims(image,0))\n",
"print(image.shape)"
],
"execution_count": 118,
"outputs": [
{
"output_type": "stream",
"text": [
"(1, 28, 28)\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAQEAAAD8CAYAAAB3lxGOAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOy9W4xl2Xke9q1zv59Tl+6qvs10c2Y4w6HE4U2yBEqBKSWA7QRhHmxBSZDIBgG+yI4TB4iUvDgPeYiBIIqCAAKIKIEUGJEsxQCJQLATSNZDBJkSaVIacTgiR8OZnul7dV3O/b7zUPWt+vZ/1j51qru6qnrq/EChzjl777XXXnv9t+//179cFEVY0pKWdHEpddYdWNKSlnS2tBQCS1rSBaelEFjSki44LYXAkpZ0wWkpBJa0pAtOSyGwpCVdcHomQsA59zecc3/pnHvHOffLz+IeS1rSkk6G3EnnCTjn0gC+D+DfAfAhgD8F8B9GUfTWid5oSUta0onQs7AEfhzAO1EUvRtF0RDAbwH40jO4z5KWtKQToMwzaPMagA/k+4cA/po9yTn3FQBfOfj6uWfQjyUtaUlx2oqi6JL98VkIgYUoiqKvAvgqADjnlrnLS1rSs6f3Qz8+C3fgDoAb8v36wW9LWtKSziE9CyHwpwBecc7dcs7lAPw8gK8/g/ssaUlLOgE6cXcgiqKxc+7vA/iXANIA/rcoir570vdZ0pKWdDJ04iHCJ+rEEhM4V5RKpZDL5ZBKpVAoFFAsFgEAURSB80U/A8B0OkUqlUIqlUIURUilUphOp2i1WoiiCJPJBKPRaOa6JZ0qfSuKos/bH88MGFzS2VM6nUYmk0G5XEalUkG1WkU2m0WtVkO1WvWMnU6nMZ1OMZlMPDNns1n/fzqdIooi5HI5pNNpAEAmk0EURRgMBsjn8/7zcDhEt9vF3t4e2u02ms0mOp0OJpPJGY/GxaWlELiAlMlk8OKLL+LGjX381jmH0WiEfr+Pfr+PbreL4XDoz0+n0xiNRl7LR1GEcrmMXq+HTCaD4XCIVCrlBUQqlcJgMMBgMEA6nfYMHkURMpkMVlZWcOnSJS8wms0mHj16hPv37+Px48dLS+GUaSkELhg55/C5z30Ok8kE9+/f90w9nU4B7AuI8Xjsmds5h+l0iuFwiExmf7rQVeD3yWSCdDrtzX4AGA6HmE6nyOVymE6n/s85h/F4jMFg4AWGcw5ra2tYW1tDu93Gd77znaVlcIq0FAIXjFZXV7GysoLvf//76HQ6SKVSyOfzyOfzSKfTyOVynrnpCgD7wkF/n0wmSKVSGI/H/hwKkiiK4JxDNpv1v9MKoFsxnU69sKHmn0wmuHLlCjqdDt5+++1THZeLTMtVhBeI8vk8Pve5z6HdbnsBoEAfBQIZm8ycTqeRTqeRSu1PF2X6dDrtrQUKAedcrO1MJoNsNuuvpxWRz+eRzWa9BRFFEfr9Pm7duoVGo3Haw3NhaSkELghlMhn82I/9GKrVqgfkyOSDwQD9ft8zaSaTQTqdxng89uY6cMj01Og8xvNoOdBSABDz76Mo8hECCorpdIp+v+8FwXg8RhRFeO2117wlsaRnS0shcAEolUrhtddew8bGBiaTCdrtduw4fX4yIa0BXuuc88xMPIBan/+VaO7zOE1//pGo+bvdrscVKAQqlQpef/11L4CW9OxoKQQuAL322mt45ZVXPDMybEeGpDbu9XoxZqUbABya8GRUan+CfSRqdHUdeJxt0lUYj8fY29vzmAHdCmAfWLx06RJefPHF0xyqC0lLIfARpxdeeAGvvPKK/05N65xDq9XyQN1wOPSCYDQaxcx94NBFYDhRTX/+KSZAolCge0BwcTqdot1uYzgcIpfLxaIBFAb9fh+f+MQncOnSzMK3JZ0gLYXAR5hu3ryJz372s14rA/tCwDnnGZOhPGrm4XCI4XDotTZ9eAA+4Ycg4ng8xmg08u0Ch64ALQS2rUJlNBr5ezMaQcGjAmQ6naLb7eKNN97wWYtLOnlaCoGPKL344ov4zGc+4/15/pFx6YPTRdBj/X4f4/HYJ/8QK+BvZGoe08/q8/M3ChGeRy3vnPN5BOoKKGg4Go3gnMOP/uiPLvGBZ0TLPIGPINXrdXzqU58CEEfnyXx7e3s+Eajb7fpj7XbbuwTFYhGFQgHZbNa7AeoeMHRI1wKAzxzUc9RNAA4tBhUWtBK0vxqG7Ha7WFtbw5UrV3D37t2TH7ALTksh8BGjdDqNT3/60z6MByAGuo1GIx8JsEQTndqb1gF9f/rzQDxqYDV0KO1XswYBxKIPFABJYcXxeIxWq4VPfepT2NnZQa/Xe5ohWpKhpRD4iNHNmzexsrIS88uBw1V/9OnnEa0EXpfNZn06MZlftbtiDvxdLQCCgyoINFVZgcBMJhMTKhQ0dFU2Njbw3nvvPf1ALcnTUgh8hKhQKODll1+e0fJkJOYDhGL7SkzqURN9Op36CIG2aU1+ChgeZ18Ue1ABwPYmk0nM0shkMjPtdjodVCqVWN7Ckp6elkLgI0IUAFy2ay0A4DApaBGia0DSCAFwGAYko9pYP/18Mj8jEsA+YEiNTwuBbsjKysrM/djueDxGoVBArVbD3t7e0w7Zkg5oKQSeY3LOYXV1FTdv3kS1WkWxWEws2kHmHAwGC7dPQaACxTnnFxNRo/OeTBdWPMGuIGRbuvKw1Wr5VYqKC6i7oH24cuUKVlZW0G63sb29faRls6T5tBQCzyHlcjlcvnwZm5ubWF1d9fF8IA646UIeMuVxhADN+X6/j2Kx6KME/D3ku1PYUAAwTKjFSQB4hm+1Wr7v6sYQMFShwvuwby+99BJu3ryJ27dvY3t7OxZxWNLitBQCzxHl83ncuHEDN2/e9AxF5gYOTXP+p6alFmbc/zikK/xseTFLZNDBYBBbM6Dan4KJRUe0P3b1obUceN9cLoetrS30+31sbm5ibW0Ne3t7uHv3Lu7du3fsZ7zotBQC55xo8t+4cQPr6+vI5/MeSae2n0wmsRV3CqbZENyTFusIuRnU5ioYNCMQ2LdM2DcVVpptSGJfbehR76uCYm9vD/V6Hel0GtVqFa+++ipeeOEFPHz4EB988MEylLggLYXAOaV0Oo3V1VV87GMfw+XLlwHAm9hkCob7JpMJCoVCzApQ5ieNRqMnNpnVoiiVSvjCF76Aq1evYmtrC3/0R38UA+q0nxoe1FTkkLYm3sDPXKxk8xDI9DwviiJ0u10451AsFnHz5k1cu3YNd+7cwd27d2dWTS4pTkshcM4ol8thY2MDL730Eur1ukfF6ZsrYEaGLxaLsRV9DLsBcYBNl/gelxSl/8mf/Elsbm7i/fffx82bN/EzP/Mz+NrXvhYLB5LJWZAUQMxKCLkTLGrCNvQZiQ+Q6UulEprNpg8tchFSr9dDOp1GPp/HzZs38eKLL+Lhw4d47733fA2FJcVpKQTOCZXLZdy6dQvXr19HsViMheiYP6/AXCqV8jF/zQ5USorZPwlRCNBC+e53v4tKpYL33nsPKysryOfz6HQ6Xugow4aWCifdg66DXsvvmo7MikRaBYmZjUw1zmazKBQKuHr1Ki5duoTd3V3cvn0bW1tby4iC0FIInCE551Cr1XDr1i28+OKLXtv1ej3PRKzkEwLmmHyja/c1GmARdQAzLsKipBGGra0tXLt2zYN/jx8/jmUiKiZBQZYUurTjoe6AJiLpZwoUuhY2iYl1EKJov2hJNptFNpvFpUuXsL6+jp2dHdy5cwf3799fRhSwFAJnQul0GlevXsWtW7ewsbGB8Xjs1/HzuGp9/dMCHbQCFPjTZbskgoe6uOe4pELgW9/6Fr74xS9ifX0d7XYb3/72tzEYDIJtH8f81iKnqvW1Hf5PpVLIZrMYDAYoFAq+Dc0y5BgyKkJhsbq6itXVVbz88st49913cf/+/YWTqD6KtBQCp0i5XA4f+9jH8PLLL6NaraLX62Fvb88zqObN24w5/Y2fJ5MJ8vn8TLkvWxJMGedpluOSqfb29vB7v/d7KBQKGA6HGAwGM8lDT0LMRVCy46HPRbyBVYuB2XJntIyIrVAYZDIZFItFfPKTn8Srr76KDz74AB988AE6nc4T9f15pqUQOAWqVqu4cuUKPvnJT6JcLqPZbOLhw4cA9hkrm83OVOOxS3DV7FeknevxLakPraT5+cehdDqNmzdv+tTddDrtXQCWKptMJnj06BF2d3ef6B7VanVmSTGAGRdHE45YACVJ8Kjw0CKog8HAb7eWy+Xw8ssv48aNG7h9+zbu3LlzoSIKSyHwDKlQKOBHfuRH/KKedrvtGUTLdZFsbNwyMYE1LQiqyTV6jgoBxQyeVAgMh0P84Ac/8MLI3jO0VuE4lE6n0Wg0ZtwYJYsNZDIZ9Pt9H3FgP3iddYtIzF5kARUK4kwmg5dffhkf+9jHcO/ePbz77rtoNpvHfpbnjZZC4BnRiy++iDfeeAPpdBo7OzveZ2b9ffX3gWSzV7EA1YRE0vUYEAf+rDtgk4qOSxoC1D4n1RU4DpVKJZTLZe/DW2DQChbNiqQVosAiSQWiFVy68Umv14NzDvl8Hs45XLt2DVeuXMG9e/dw//59PHjw4CMbXlwKgWdAr732Gn76p3/aL3Ah88/TwnaCWeZXog9MBFyvsaCYjRQoiHYcIpMsQk8Sfms0GrHxsZEGCgTmOmh2Id0jux8Cr6dgVGGl/+kW0DIA4DdXvXLlCq5du4YPP/wQf/7nf/6RFATLGoMnTNevX8dP/dRPodvtxvba02Wz80xmNWn1WhJTf4kjJFkBQDzvH4DfgTiXyx37uShgFvl7ElpbW/OCi+BgKF2Y/0lMQebfPGBSsyitBUZgluCs7pbc7/dx9epVXL9+/Yme7bzTUgicINXrdXzxi1/EYDDwJmoqlfJAmi3fZRnGam17TP9rUk0IBNTkIBUU+XweGxsbJ/vgT0npdBrlctmb+KGkJi1xzuedTqdeoIXAQQskqiCz46yfmYHIpKN+v49Wq4VXX30VpVLpWQzBmdJSCJwQOefw2c9+Fs65mRp+LMulyTyhclvz2gYOE4QUUAxNfCCedsvvdCFKpRIqlcrJPfxTUqVSQS6Xiz0TmT0pZGjxCSsErFWiAsHucaDt8Fy6WpqZGEURXn755ZN89HNBSyFwQkTfkQKAf2Rw9VP1OOv6qyAIxfh1spJhLBKu55N0SS/bGY1GWFtbe+LswZOmzc1N/9kWKQHi+Q02EqH5EXQH5qVHW3dDhQOtN72+1+v5RKLBYICNjY1zZ0k9LZ2PWfCcUyqVwuuvvx6r06+aiBOafqdlPquFQhqMyS4EGe2xEEJPgaDalH3TlXhnSfl8HleuXAFwmA9ARrYAoC06YkkxAfsOrJAFDi0ru0eihiBViHNh1K1bt86NAD0J+ug8yRnSxsYG1tbWYprImvg0Ma0gUIRbNZMNH3JiE0G3Pq22oRECAB6k1G3DxuMx6vX6mU/m69ev+7oEw+HQM5zuOxBKpLJ5ECHXSi0sG5bVYxQumn7NEKIuZKK1UK1W/fLujwItQ4RPSanU/o6/nCAWjOLkscCd+uoAZhBxMjWJbXKprabDsh0VNMPhEI8fP8aDBw+wvb2N9fV13LhxAwBw+fJl7O7uot/vo9FoYHt7+1kNz1yqVCq4ceMGdnd38dZbbyGbzaLRaGBlZQW1Wi2W4afMreNJocZxZk1EHVe1lkL5Ago48l7ZbHZmOTOtseFwiJdeegkPHz78SKxGPFIIOOduAPhNABsAIgBfjaLoV51zqwB+G8BNAO8B+Lkoinbc/tv5VQB/C0AXwN+NoujfPJvunz3V63Wsrq7OFNXU+LZqJk5IW6ufkQQ1ifV61ZAqUPS8dDqNfr+Pe/fu4e7du2i1Wv7ezO93zqFcLiOTyeD27duoVquxOn+nRel0Gq+++irS6TS2trb8Tkh7e3u4ffs2isUi1tfXcfnyZZ8ZyaXTNmORpMVLNOfAjpcyrrpK3Jgln8/7sWRUhy4UXYhqtYqNjQ3cu3fvWQzPqdIituAYwH8ZRdHrAH4CwC86514H8MsAfj+KolcA/P7BdwD4mwBeOfj7CoBfO/FenyO6detWLJXXxtOBQyanNrO5APOy/KxlAcRNf2qwfr+PH/7wh/iTP/kTvP322zMFNHZ2dtDpdLwrcvXqVX+sXq8/0zGy5JzDjRs3cPXqVYxGI2xtbcWOR9F+paDbt2/jz/7sz/DDH/4Q7XbbP6tGAnQMFU/Q3ZdtJEWxEroDXE/AMaVQJ+m5LNh67dq10xiuZ05HWgJRFN0DcO/gc8s59z0A1wB8CcBfPzjtNwD8IYBfOvj9N6P9kf7XzrmGc+7KQTsfKcrn87h27doMCAXEy3Fx4nIiqTVg8+FJNjSmmW+6NdhwOMT9+/dx+/ZttFqtxL6ORiO89dZbeP3119Hr9fwCmvF47Bc1ndZy2kuXLmFzcxOtVgudTmduLcDRaIS7d+/i8ePH2NzcRKPR8GFYW3sAiEcXFJi1YUF1IwB4UJegKy0jzUDUnZeHwyEuXbqEcrn83K88PBYq5Jy7CeAzAL4BYEMY+z723QVgX0B8IJd9ePCbbesrzrlvOue+ecw+nxva2Njw22pTc2gxTWAW7eeuvMBstRwStZlqLq3Tn0ql0Ov18Fd/9Vf4xje+gbfeemuuACD1ej38xV/8Bba2ttBsNn1qbDabxerq6skMyhG0traGW7du+SSfDz/8cKGKR4PBwK/wo7nPcKwNl9qoiI6j1mrgOOv+jIwIKAjLNQaa5EVL46OQRbiwEHDOVQD8XwD+8yiKYkurDrT+sZKqoyj6ahRFn4+i6PPHue480dWrV2PbddNfZC1ARaS15r8VBJoYo6Yt26WlwSy2Bw8e4Dvf+Q6+//3vo91uz000sjQYDPD+++8DgM9+y2QyKJVKKBaLJzxCcVpZWcEnPvEJXy59e3t7xhU4ira3t3Hnzh1fbZlC14Ze1SJIStfWPA0mdOkGKNpWp9OJ4T5MJ7506dJTjMj5oIWiA865LPYFwD+NouifH/z8gGa+c+4KgIcHv98BcEMuv37w20eKMpkMGo2GZ2jV5N1uN+br85gi1danDYX2NOGI2v+9996LaUNuHR5alkzSrDr29d69e968Zs78ysqKF2AnTSsrK/j0pz8N4NDUfvDgwZGJPfyvGZeDwQDb29tYXV312tpaXvx9Mpkgl8vFQFXFaHgOhSwFAgVVJpNBq9VCt9tFrVaLuRrD4RBra2vI5/PH2tTlvNEi0QEH4NcBfC+Kov9RDn0dwC8A+O8P/n9Nfv/7zrnfAvDXAOx9FPEAvvxerxfz84H9Sbu3t4dGo+G/A4d195XxbaKPDX8p0HX//n1sb29jZWUF1WrV59vbYiNa0VeBL7VA0uk0Op0OCoUC8vk8crkcarWaDy2epCBYWVnB5z73OeTzee+2dLtd7OzsxPAQWk8Ubrlczi+UInMqcMd9B6wgIGmqNM+h60bNz/sOh0NfqYihWOccut0u9vb2/HoCRjE41sRUPtJCAMAXAPwnAN50zn3n4Lf/BvvM/8+cc18G8D6Anzs49nvYDw++g/0Q4d870R6fE9rY2IiZk5qOmsvl0O/30W63Ua1WfeKJgk2WVBBYBmS7m5ubuH79uteIwGHWG4BY+SwVBJp8Q4GglgetinK5DGCfmR49enQigmB1dRVvvPEGyuWyv3en08GHH36I0WjkU6BppeRyOeTzeRSLRRQKBb/4iucwTs/di9rtti/NbtOJKVB4fbfb9YKEGItzzjMwx42h1OFwiN3dXQyHw1j5dy1lNhwOzzTX4iRokejA/wcgaSH5zwbOjwD84lP261xTKpXCjRs3YmmqjCMD+wydz+fRbreRz+d9nJsMTmFQKBRmgEHVThpSBOC1k+YZMLHFJsAAcUAROAS42Aeuo9clvFzMA+CpBcHly5fxyU9+EoVCAYPBAN1uF61WC++++y46nY6vAqwanrX/SqUSCoWCLz6q4zedTtFut319Rmp6mvAUipoENBgMfE0Eiw2wZBp/4zh1u100m02Pl4RA38FggNXVVbz77rtPPE5nTcuMwSegfD6PbDaLfr/vk3xsmjAn5s7ODi5duuQnJPcQYFipWCzGBIAyvW1XtZeNLCiarUzBfQFVcPD/cDj05q0Kl2KxiKtXr2IymeDx48dPNEZXr17FJz7xCT8GTF4iwJZOp/2uSdTa+Xwe1WrV77BMrELxDv7PZDLeF6elQ2HI9oiX0FriGFEQMBuRwkdThYk7RNH+RieZTCY2VsCh5VUul2ObrDxvtBQCT0DVajUG2gGHPqJms2UyGa+xLD4AAJ1Ox1sNulJOU2WVNN6tDM0Jre1zwqfTaZ+Tz/MI/kVR5LUjr6cAyeVyPs34uILg2rVreP311+Gcw4MHD3D//n30+31vjgPwjMWKyYVCAeVyGZVKBYVCwWv/EMDK8SVewHMZPaFAUyGgqzVtgharPtGqmkwmXmBVKhWUy+UZC4Pvn++8Vqs9scA8a1oKgSegWq3m99NTIG88Hnsfl5Msl8uh1Wp5M5dMTP+Tqb0qCFTraUqxhgxpvluQUXfjUVOZgoAMMxwOkU6nvYlOl0GFCndBBhYTBM45vPjii/j4xz+ObreL+/fvY29vD8Ph0BcEpdBTXz+bzaJSqaBYLPqUXcVILPOHBGRIaLCd0HvSRKNMJhNbRTgcDrG9ve1XWtLqI9BIy0vzDFZWVpZC4CLR+vq6Ny91MlIIkKhdCDCpWctj/X4fe3t7qNVqnjGAONKt2k2BQCBcWlzdBfW1mRFIRmF/2u22D2vyOQqFQkwQECxMIuec30mJGX7UlMPh0GMQhULBRzaogQkA0vxXZlcBoEKRv1HIKfOr5UCNTbxA8wV4DS0IAo+7u7sYDAYol8seLCVuoQJKgdXnueLQUggck/L5PBqNhi9IaTWMgnoA/CRnufF6vR5LFCJoxjr3qgk5mamp1B3QcGPSfw0HcsLTfGW7zDHo9/vo9/uxTUSI1mcyGbz00ksYj8fY2dkJjsuNGzdQqVTw9ttvx8BIZuNlMhlUKhU0Gg2USiWPq2iug/ZfsRElO776bEB8jwK2E3KrKAT4/nh+p9PB7u6ut1i4mpBWjOYfsC0KAWtRPC+0FALHJC5x1QUmNr0XiJf3IgDGaIGi4pxUg8EAnU4H0+kUhULB57Arwypp0or1mS3QqLUGCZ5RIJAJs9mstxRoRdj9DW7duuXDckrr6+uo1Wq4c+eOBzrJHIyjNxoNLwAoeOj+KB5C0mfQ8VQGt66THufYaLsWZKVwUnyn3W5jNBr5qA6vo2DTVYUqcJh1+TzuU7AUAsek1dVVPwnsxCX6r7X9qdVLpRLG4zGazSbq9XosqsDwolYlKhaLsYrCNN85uZWxiQVomJKkGlVz6ZXJKAj6/X5sEVEoFPnSSy/hL//yL70l1Gg0kM1msbu7i3K57EE4u+9fpVJBPp+f2UHZ4iChXAkFX7Xv6terIFYhYYm4ioZN2bZGSxi90AiNZizqwiK+k9XV1edSCCwrCx2TyMA6kXUFoSbuKLOmUinvN1LjUxvbBUidTgftdtuDUQr68Y9+sOaysy8KqFlQTQUK26DLUqlUvFnLLDpbc69YLOLjH/+4FxylUgl7e3veXWG0IZfLoV6v48qVK1hdXfUYgDXdVdDoZ5Iysi0vpvF6S4qJ6PtgO7RyOGaTycRXFmZ4kcKcx/ndChIeZ0LR80ZLS+AYxKw66wpw8mpOuoaTAPj89Uqlglar5SvgqK+uJjhr3gPwPrNqfN5bw1T0WTWN2YKE1gKg2c5Jr/1pNpsxS4RMV61WcfPmTZ/7X61Wfc4EBQkz/wj4KXNoZMOa8/ysTAbEsyy1T3qtfWZqaAVx1YwH4AuV0B0jKEosxDnnrSP2h+9WrY7JZOKf9XnLF1gKgWMQN+7o9XoxcxCIb4lNX7PX68Vy3JkKq2BZPp/3E0jXq0dR5E1uZs/pQiEKDQW+OPnI3JZZGCpT4JGTNooiLwQICHKzT11hRwZiNSXmHBQKBdRqNe/zE0fQa9hfC/qFXAE915r4Fh/RqIFNqeZ/xQNsngB3HmL6MN03An0ETCkcNVLB8eT/fD7/3AmBpTtwDFpZWYmh7vpfAToyDVewAcDW1hZ6vZ7PyFMNBRxGERgrJxP0ej10Oh10u91YiE+Rdd6L4Ti7vJnX2HXxZChNTkqn0z5br9Fo+AVGBNDIUDxOS4Ltc9GPLv6xWlvBwKQQZ1K6chKAqKa+BWoVP2E/GZ4dDAaxP4ZFuV0bBbaGfvUeujIxiiLUarVFp9O5oaUQOAZtbm7GJoBqE51s1NL8vVKpIJ1O4/Hjx97nLBQKnmGpKclEnIRkIu6AozgB70+G06KcCjACmBEE/E0z74BDRqJwoWDKZrMxoJIChOFSdWM0oUYtDmv6WwHA86yloDkSmkNhXQF9TgplxWpslIB9ZdJXt9uNCUEuK+b7Yt9oGRFTUYtlOp36zNDniZZCYEHK5XKoVCqxZJMQccLQNB0MBn5hThRFaDabsQo21NrcuoxhPCbQ0CoYjUbodDrodDqxWnia5qsRBF0taBkNiGfdMUmHRLeDgokChj4vEF95SMBTQVAySEhzq/C0Jn8SThASvJaUGa0roKCi7jNAi0BNfoY5WQiG48PfkvrBqI51V847PV+9PUOqVqvef7bJJyEUnkzCNfs0vTnZ6Dtrrbx+vx+zADSzjxOXGiuKohm/WzECTnwL6rEf9G0pTCxzUrho1p7G9weDgUfMmTev+IGmH+u4hASDClQVsBZ8DYGbGg3RJCWStdDUCiAxLMgIDt8Xcwk0mYkumQpO7W8ul3vu8gWWQmBBqtfrscmqE85GCDQRhyFAncCsnEtLgefSOtCQIHCoWclkjB5QEFBbA4hpPRUE7JeCmRYzUNOZYT6ay1p6TNNsGU8vlUoYDAYebQ+Bf0nhMytAk8i2qddbQUBTn7UHmAdg2yIuQEYvFote6NL94m/D4dALbc0vUAxjOp2iWq0+V0Jg6Q4sSKurq7EQYGjCqq+umo+hJ/rw0+nUV/fVeDXP0dRTLpGlyU+TnP5qr9fz6L7mEKj2otZXmgew8fk0eqHJMrn7lXEAACAASURBVGQWJtPQLWBuwTwhELKaQkJJx9O6MxYMBOKuSBRFHvFn7J+WC/tHRu/1et7U5/jS7OfCL4KEDINS8PK+NlHpPGzvdhxaWgILUC6X80g4/W0gDmYBsymuTLltNpvedOd5NO1TqVQsZ5+MR1LzVv10BbYAxJbf2r6pr6ymNC0EC7yRGDYkE5B4D91JmElD6pMv4sfb8CA/05VQQWBdAeCQ+TWbMooivx6Dqyd5DtO1rUDgGLIdWgFcRUjsgEJW37MKf80XeF7WESyFwAJUr9d9br0CWDrpLPpMbZLL5XysmSg8fW1dzEPhwpRbBdeAWW2pk16TkzQuryZqCGwD4ivukoC4fD4fEyTK1JoiXSwWffTCRgSsgEzS/jzGPiuzW0FGwaWLecjcBFCBw0VG+sx0Dwi6Eouhi9ZsNpFOp1Eul5FOp/3uxEwiolWorqBiJ/V6/blZWrwUAgvQlStXZsJOFhy0pPH0VCrl6w1q6q9msQ0GA59IRB/fhsss7qCMCRwuvQ3VBuD17FsIsFOzVs9nX6zW1r8o2q/Aw1yIeRiAkobYtC29jx6zVgWFKf90/YMConwGfmdGJoU4hcB0OvVWQKlU8kVGiMHkcrlYXQfFWbTf6+vrz40QWGICR5BzzlcSAg6ZWwGhkFmomX3UkPQpdUUaGUVxAfqvvH9SrF1DhGoBaDgsdH0SYGfPteNgj/E/hRbdBpuRZ9uzLovtFwVZktVCItOzMEqr1Yplc9p+0IwfjUbo9Xqembmyk4u4uEpSzfputxsba7VeLM7xvIUKn49eniHlcjlfKdeSmoLAobYBDicGhcB0ur+BhSaZqImspiUTVNLpdCxTjRRiZP0fOj/pd+ve2Fj9UW2RialFNWwWEhqWkdW85zkKvIZcFF5H8I/JUUB8nQVJsRIK8Ha77QFUrhUA4LdoI0hIEJdLommJqcC3QCWFWKVSwe7ubnAczxMthcARtLa2FvM71YS3LoE13flbKrVfvabT6fhYtC70AQ4FATUNU1iBQ03L9kIUEgQh9D3pGD/b69XE5TgoYKfPzUVINlMwNC62DR6zWY6hvAEdL46PthuyQiiQiQVQ6NINULB2PB6jXq970JZ1IBQQ1HwI21+6ZdwC/rzTUggcQZcvX55ZOqwRASA+OTWMpig+owTdbhfO7W8Pbs1wakBmDtq1BqrhQpramqUkC1yGkHj93fr+et48LIQuQeheyuBJgkD7bMFNe29+pnBkboAyvQohxviJ8pNZKdQA+LLkTMWm9QYcVnxiBEDdDYulkFg0lQvBzistMYE5lE6nsba2NrMYx6ahauhPEWxqcIaNiETThCXTsz3iAQS5CEipyavMGGJgUpJJb33ZEFmGS8ITQqE/KwRse2rqhwSKZSR9VhvC1OdgNEX/2G+NLHD8SZo+zKQihnOJNyh+QyEQcgG0/3wfz8NehUshMIdqtZrXwkA8JTgUA1YXwZqzTPgB4KvvqmtBzbSzs4O9vT0vJAhUUQhocgop5CJY7aT9UnDT9jkk7LRAihUilpEVDEuKqOgz2z6HLA/FLOy6A56vtQbJrLQ0NJeAGZAkLTAyHA6968asTFpgdAH6/b5//2zbgrjsGwuNzMuTOA90vnt3xlSr1WIvWOkojQbMal1drWetAWB/QvZ6PV/MQ9HsTqfjzVn7p9ZEiFG1P0nmtWVIndDMX7AWyI0bN/wqQpLiF6F2Q2NkmZ/PFYpyAJhh7tCzcpzJsDymqcN6Ht0FviOGGilc6KbxvVFo8DvfjfaJkYeVlZW5Y3DWtBQCc2hjYyO20MbiAPMmuGpQ4HBCkDhxlBlzuZyPQwPweABBKy49VqZXptHf9Xio/6QQHqDalNqeayA48cfjMa5evQoAMeGkUYYQEJnkzliXiwh7yKTX9jTawnvz/dB6YtSGaxvs8xMzYP/VUtN7UqBQy2uCFjEEnR98rvPuEiyBwQTKZrNYW1uL7dZDUsbT0FMSaMcJocVCoijyG3JykpLRaRFwQw4FsbgYSYtZAGG03faX51jsQM1tFQZqVmtbwH6RlN/5nd/Bzs4O6vW636UZgI9sqObmtcpUem8F9XTMrMWl4ViG9SgklBRXIS5DRlfitXQFAMSELYAYtsN3RZyAhWCsBaCWzXkHCJeWQAKtr6/7SWa1bcgXJqk2soLDmsrUrpxAnHwAfDgxlUp5sI3CwGp5Usi3D2nPkFWgGtheR7eEUY9UKoXd3V1Mp1Osra3hwYMHvsYB2wq5IVbT29+S3BiLJ5CS0HnV2GTabrcb20FaSSs86WIjruPQKA/vaxd02bEHDvGG6XR6rl2CpSWQQNeuXZsx15VCeQLzzuGEJOjE4yw6YkOAg8HACxtqfpqgNH+pmVSTKiWh/yTLtLyXhitDoFY6nca1a9cwHo/96kq7yMjmB+j99LM9LyTUaJGoiW0Fso61anFGWBjqC0U4+NzaBqs7scKQVk3iQiRGEazrZ9umELh3797c93FWtBQCAXLOYWNjw4eMQqTCQYEqTeqhhtEJns/nfREL4BB0Y1iL1yvAp/4uqxBRMFDzUJCwP7xOGdn6rCEtaoE6LX6qUYxqtYrPfOYz2NrairkoylA6ntofy/AhgZFEVpAQp9D3oFbDaDTCw4cPAcDnZiQJAf5ppSZreQDxXZ+J12jFIY1SsJ/c3rzX6x35jKdNSyEQIF0lRlJfOQlk43lAHBhUwcDltxbA0pLfJB6j/2nBQLapK/l4HYCZ/oasBZ5vJ7wuQiLzq489nU7x1ltv+ZV4dkyUqRQw1OM6XqGIArWsJvTouA2HQ1/RWesn6HOyvDutJzsGdpEVTf2k/nAVoS4pViDXkgralZWVcykElphAgHSlWCiMpog5MKthNcRlzVuakkoadyZxMioTadKOvUcoWpEkqELE9nlvq5UVg+DEJoJ+VNKRTTSyn9VyUcFlQUH+RiCP/jvDe/Z5R6MRdnZ2fJ/ZLt+bjqUKbF2MRdcilUrFtpBTi41jBhzmK1jrhq7TeaSlJRCgarUam4CcNDZFlUATEC+EYdFw+1n3/eMxKwR00QvXGVjtRHzA4gkkMqii9FYj83c9HiKbCKNMrJYHx4saV9N/1ZqybfO/MjqTejQxS4WADZdyHGi9PHr0yLehJrqNTqiQZVIX3Ruep5uR2D7baIt1IdkOhch525dgaQkEqNFozKDv9IWBw8ljhYBdqUay8XOtyENSv1KPaYER5rSr5lT/12pLJWuKhxh+Ub9c27M+unMu9hwWYwjhAZaRGColo1s/n3sEqHCwQomugh6zLoViKgT6yOxkeE1Dtsyuz51UhVjH2Tl3LkuSL4WAIeec3z7cmtgKwjnnYuv+Q+fSzLQJR9bnBPa1diiOrOYlcFjJJ+RLE4egH61gXoisZk7S0Nq+HrPAHs/RXAQVgNaSUFeAxLX+rAyk6dJkNH2epPFnQhPPV1yFRIuBSVqaeqzYAK+zFg9/Y+ZnSMhYAbS2thZ8F2dJSyFgqFQq+T0CyMjqeyuCrPFlgljKGEraDoBgnQBaFhZdVtJJr/cO9TdkEZBCZrkVYqHnsGCp1eiKtqu1YMFDPZ8MwsgH03NVmFmXwgpHXq8unD5Pr9dDu92esR7I7No3KwyU6UkUOprbYc+37lEURajX68F3f5a0sBBwzqWdc992zv3fB99vOee+4Zx7xzn328653MHv+YPv7xwcv/lsuv5sqFKpBENIQHw3Wvp21N56Dtea24KUwCHjhFwC+sE6mVRbKkqtFMpZ0NCVTvx5YGGSRWCvswyZdK4KTWt16LVkEFoBZGCbpquLdXRMKOwolAni6ThNJhO0Wq0ZEFILweg70yXhChbaNgF4N405BWxLx0nDxdys5bzQcSyBfwjge/L9nwD4lSiKXgawA+DLB79/GcDOwe+/cnDec0MrKysxM9qG+LgYhZV0tPgoEGdQfqYmoL+q69EtWa2SBDLxszKVWgKkUOTCamP9PE8Q2HtakFEZMiQ0QqShTnVhdNzs+FoAVs+hAM7n86hUKjP3ovDR8bWLxJLcIvu+dByJ1VCwqCuo1tN4PD53uMBCQsA5dx3Avwvgfz347gD8DIDfPTjlNwD8Bwefv3TwHQfHf9YtijadMaVSKayvr8eW+AKzpjHjxQASw2SK2Gs4UTVEyCxU3EDbIqllYMEp218bqeBveh/LnCEBwedQtNxeE+rLPARd72UFmAU7ta1QngStAA21ZjIZ1Gq12KItAD6vgS4d9yG0iD2PW9zCWi8kDTOqMGQ/+d6n0ylqtdrCAOxp0KKWwP8E4L8CwNm4BmA3iiLamR8CuHbw+RqADwDg4Pjewfkxcs59xTn3TefcN5+w7ydO5XIZ+Xx+RpvqpOWLJWikPmFI02oCi/rGmvKrpMdsP0Lt6u+2v+o7A4vt48fP+iwK+qmVlKTdbXJPCF/gPVQYUTNTe6o21bZJOrbqLnD88vk8Ll++PCMINCTIXZZYs8GOk2I99nltHoPWGLACWa9hSbPzQkcKAefcvwfgYRRF3zrJG0dR9NUoij4fRdHnT7Ldp6GNjY2gf80Jo1qAqaVMWgkBUvQp1RrQyWGZWePqISZiG4oVWEFiBYXNQNTP1hpQDabW0LwlyXaSh1KR9TlC7oq2qZt2qL+ubYUsDM0Z4PX01S9duoRyuQxLxHaIn7CICJ+D/bOgsD6PWiF0R4gb2bHXeVGv12f6c1a0iCXwBQD/vnPuPQC/hX034FcBNJxztFmvA7hz8PkOgBsAcHC8DuC5KMC+uroa23DUxv1VCNDvnE6n6Ha7MQ2loSTG9507LHSpjKsagb6lgmmcgBbtZv94XihEqf8t6bOpELEhNEsh89ia+qH+2GiBuhB6nu7zx2vUzA4lPCmwqEKMTJ7L5VCtVoMxfL1fr9fzJcvZrs3d4L1s1EJdDItV2H6ykOl5oSOFQBRF/3UURdejKLoJ4OcB/EEURf8xgH8F4G8fnPYLAL528PnrB99xcPwPoqSZeI4ol8uhWCzGYvo2UQWI+3Zc4ktEm2SBLMagCSRSiNhJQyyA26Db0JQym2IGNgxow1R8Bp2clkI+qt5X0fQQE+qzKxDK3ywpQ6qbpVaQ9l8BtpBVoMyvmlmzLjlm9MkpKIjNTCYTX16cbYesLYsZUdDbBWBq+dkxslGos6SnSRv+JQC/5Zz77wB8G8CvH/z+6wD+D+fcOwC2sS84zj3ZWnCamqqJI8DhxONacm4MajelYL0ABfq4TZdaABQQFC7Xr1+fQZCVkdUdUFeBfdPPSfkGVhCouc5rLeBH4oInCzoqc1jBFWJc+11dGeIDFBC6JbhewxCe5kVomFaFF903bgPH8mF8lul0il6vh1arhXw+77GEpPGzAK3NFaFbQCFiU8Ft+vhZ0bGEQBRFfwjgDw8+vwvgxwPn9AH8nRPo26nS2tqaB4YYr7fIP7U0GSSd3t+rrtvtotPp+C2safq3Wi2/vRhfOgC/RLlQKPh2eO90Oo1GoxHzjanVFCG3ITIboaBAsqa+DT2GTH5lRmVeXmezHUNMoowdwixoKYQAWHUx5kVp2Cd1ERRQVIxlOp2iVCqh0+mgUqlgb2/PCwsKCd6r2WyiXC7HisNq/3UMNa+Az6Pt5nK52A5Hii+Uy+VzIQSWGYPYZ5j19fWYz6ar0waDAfr9Pjqdjt+cghqHu9J0Op3YWoJ+v49Wq+XPpTVAQaBr4GllMG+9UqkkMpplZItEa4KLLge2CUiaDhvSzKrdOCZ6L+2T/c82Q8wf8pFD2IHeX7EBdbt43NYZtCAs2ygUCqhUKtje3vb3Zx9sQReNFpBoNRBn4F4EWltAiYVJtEAJrwP23ZLzQEshgP1Vg9S8rCSrKbyKEVDr83ihUPAbiXa7XUyn05gVoSgywURgnxkGg0FsbXo6nfY76dJftclBQDh5RRlA3QUgDmxahg1FD5KAQh5T39xGDHS8tF8hLc/zrbBQIcX+0TqzERbN7ycAOy/y0mg0kM1m/Tl8x9TWbLfT6aDX6/njurKQZcd4jS4u4jxhshKZX90CCvBisRiekKdMSyGAfTyAmm44HPrJphNbTUtaBmTYer0O55zPTbfIMTWoamkAvqgoY9qafWiZC4hrUcURkuL/VsMv0q5Fuy1OoNZMtVpFo9GIRQPUHLbRgJA1kIQZq7DQMWT7JLVwVBhQIPA4xzybzcYAXwpH1gvkOx4Oh9jb2/MrEdViU2bW77RA8vk86vW6zwewz9HtdtHtdoMLyc6CLrwQcG5/eaea7fxdtZSdxCxrRZcgl8t5l0ETgax25QQFDpfF8jj9WH63PrOCUdoXG0Kz5+j1SWCfjofFAWzuPBlyZWXFg2zWotC4vd5T+23vpeMU+s2uH+AxFayh+LwKDh0fvkNNBSfmM53u7w3RbDb9O9IxUeGiy72LxSIajYZfu6DChpZks9n0wsXWZjwLuvBCoFgsolKpeO2vWkzDPKoJOCm57t25/e3LoyjyOABwiNSrJrQmPqv08ppSqeQnsqbIJpnXQFwbhvxwJW0riRGV1K0ADhknn8/jT//0T/HHf/zHfgMO9kX7EEL+593XWjvKZMpQfD88T/t5lKDTDE+a/uw/IwVsnxuScqGYRiXUpaKvT0BRIxXsU7vdRrPZ9Much8PhUgicB6I5a9elqyuQz+dRrVZRq9XQaDRQrVZjcX8AHsyjq6CaXN0DahyL7KtFYH19ADN+uC56seY9r1/EIrAMyftbH1+vA4D3338fqVQKq6urePTokX9m7bP+t30IIf0h/EFNenULaH0odqKC0ApPK8jsPemKUdjz2Hg89vUNdOs4GyXQ96Q5DRT4zWYTjx8/9jklFAI2pfks6MKXF1tfX/eZZuorkjmYuKMVfRgaYpip3++jUCigXC6j2Wyi3W77cCHDgcoQGpemG6AFMELJPEmhvKR6ASG3gfe3bek9LcZgtS8Ab/ZqUpAuo7XnWwvFRhFUQLLvNoSpoT7FH1RI2PoObEMFhB0zunNq8aXT6RlrcDAYeBPfbniqQo4CRMdxZ2cHjx49Qq/X81EKPr8NQZ4FnX0PzpDS6TRKpVKwlBWZleiuIs8A/EalWs220Wig1Wr5ZBogDmqROJHG4zHS6XSsAq3FAUIaU4nMo4LGmvuWlDn4XcHAUD4B+8DfGo0GXnjhBRSLRXzve9+L4RzKoGoBWKslZCEoAyr4pgKY2Anj7rb2n2ISvAcjAvZd8Df+WRBYoz3Wz7cRjCTLoNVqefAXgG9HQ8pnSRdaCFQqFaRSqZmilZw0DP/Q/FSpTf8+m816F4Dhwl6v57ECnqsTPpfLoVwuexTaouUhv161p9Y3sEi5Yga2nJZSEiqt97dxdisQHj9+jFQq5X1gzUuYx+h2XOzzhc7XrcA02mKFnFo0VpCo1WGfmWAvU4j5DGRqhoBpAdB6U6GhUQi2MZ1OUa1WY5Yen2c0GmF3dzf4Hk6TLrQQuHTpkrcCOHmYeafhG75sRXnJ5Pl83ocWM5kM6vX6zFoCTly6AZo5mESKatuJBRztNlgz2wKJep+kz2rqhkx1FWBs0xY6ScIFrNUyjyjMKGwIpGr/uOJQ8xSsQKXrlUSs6mTBV/5nEhG1v3VbaMWQVEhzH0oSsaZMJoO7d++eqUVwYYVANpvF5cuX0Ww2Y2APEK/wy/RbCgeddLpSjag5rQEtGqp4AwUKs9YsWdOSfdOJabWynq9kJzHPs35s0jXK+PxvkXhLyni2ZkLIOrCgpu0Hx4CWDb+rZabgoAoHvcYCrLbPtAi52Quv1X0GKGj4ni3uwvqIBIyHwyG63a6PAPH8yWSCYrGI69evY2VlBeVyGe12Oziep0EXVgjU63VMJvu7/3JSWTSav+l33QcQOFxPQPBoMpmgXq/PbNCp4aBWq5Uo+avVaozJtQimRdKteR7SrtZcVvR/ETOd54WsDvZT8xf0GtsXbVtBRApU20f7bIoD0Eqi1QYc1h9Q68Um94Rq/qsbQOuJWpp4EP14C1DSuhgMBn7PQ1ZJ5v25QpVtTqf7y8+3traQy+Vw/fp1vP322zP9Oi26sEJAF5GQqP0p5am5SXQR1FdUEI2+IlNK1RpgKnK5XEar1Ursl/r71gKwCUghIRBiZKWkiIICjMDhIiUyFJ+VJrNlfO2HAmf6+zygUgFBfTZ1M/h+NIU4JKxJof6FhICG/NR90sU/NOcVI2JBmXa77XMNNGFIGZ/vjuNTqVS84FhbWzvTTUkupBBIpfa3+1ZfX1+Wgkg08XTDUE44nsP2NLe9VqsFrQGrwS2RWaz7ocdIFEh8Jm1Dn9UKC1KSxaDPp1YPmUD/k8jIajlov9kHi+Ir6q++vAolPgMFhQKeKmx0i3C9l7aRhAmMRiNUKhXfHrMHdeMRConxeIxms4lWq+V3O87n81hbW0OlUplhfr4HjU5QoKlFc1Z0IYVAuVzG5uYm+v1+rGYAgJi5Sa3DF88JVKlUYkt1NeRHjZDNZlEsFoM7EM8TAsBhKqwy37wcc04g9b/1GhUQCjSqdlYBo66OXm8BRhUiNj7OMbTWgLUIbLiO46nWigoQChp1CfibrtfX/tnoS4h4D5YD14VBZODBYIBer4e9vT20222vPGq1Gmq1Gsrlsl82bpmaVqIKbX1fZ0kXUgjU63UUi8UYuq/gnU6kQqHgY/msi0+XgRqL0QIAPmQYRRHK5bIvIgLAF7Q8SgiQVNNzMqqGpcbl5Gd/1Iy3jKfH1G/WFGW1HhQPUERc/f6Q/65AWAgXsMTz2ZeQNaPHdQGV9kf7bl2K0FJkpU6n45eGq1tFa5Dan0qjXq+j0Wj4+aTpzTYCowAviVGis96b8EIKAc0dBw7BN/XzqW20OAQFBcNEmjegvjyTQLgyTa0BAkZJxHMVHLOmtAqCpIiAZQAbKTgqJKXXaT/IAOq7h/oQAgz1viHtrNiKRmX0+dUasOCmCko1vymUjlqxN5lM0G63sba2FrM0BoMBWq2WLyjL+hOXL19GqVSawSJ4H7qKKoApCHgOt1krFApnVmDkQq4doDbmC9bFL0SEieryj2nBLFFNcI/SXBe7qCnPSUIKJasoKZioGgnAjNYjc9i8eWVSYDZykIT222t4nprmoaiCWg1kOhVU9vi851dBwHNDQkYXFmk/NQvSgqSaD5JEnU7HWxk0/be3t31tgVwuh83NTayvr8e0vxUEFCKaNKb95j1oNZ5lbYELaQkQC9CtxGyhSH2pnGhEixkKarVaqNVqPsZMwCyfz3vXgkJEtyubxwTK9IpV8Fo1dUO5AcqIVitaK0CZk6TChMcUebd4gt439NkmIKnFxP/2/jbphs9AJtfIhb2HPrd1GRbxv6fTKXZ2dpBKpXz4mP2jAOACMvse7ZjYSIX2k9EF/naW+xBcSCHA+n+sK0hpzBeoy33tfoJk6mw2i729PaRSKVQqFb/ajCGlQqGAwWDgIwd84UeZ4SR1TUgWuLPoesg94TEL0Km1ooyifrya6ZaBQs9hXQdeY8N8ijtQaOl3bUeFl5r1NMvZb3VT1OzOZrMzQuMoGg6HXrgzB2A0GvlCIRrqm07jRWPVZeEz6ypS++x8v6F9EU6LLqQQAPaXdq6vr3sLgExqUXXrz5HZuHOxLg7p9Xro9/teUFCb6QpEUpIwIJ6g99RrQowS6qv+rteHtJW6FcBsBGFef0OxeI0UaDuWma2VoEAlLQ9rtej1aqHo2DD0pqFdG3GYR5PJxCf4UGMPBgP/HlVJWDdEQ7ycLxYb4TjoOUfhFc+SLiQmABwW86DvD+xbCCwbZies1U6pVArVahXVahXtdhvD4dBXj+31et4CIKjIiXPUttRcd6Cr03jPpByDkJmfFM5L8uetcLCkgKm6JCFT3N7TauBQToL1p/lbKNRmr1PrwwKi+nyLAm8aYdE8AY2u8Ji6jBSojEJQYWi+Aa0YjVSEBOlp0oUVAgo6ERAE4LPAdHJrcUtNFspkMl4QdLtdOOdQqVTQbre9IKClQS3CRTDzSE3mkHnOc6g17ZJgns/f9L+9D0kBOE7Uo9wXakX1zSmAQgLJgpPaTw0LhrAO7a8yIsdAwUh16+x7W4S0nyHBomQjEfzT6JE+jw19auTprOjCCoFms+lNb2aHAfBFQqyUtouMyAB0DUqlkscEcrkcdnd3MR6PvRAgVrCoSUpSk5L3pgbixJ7nu4fa4md9PiBuLXByJ2EBvN6uVFTLKdRu6HrtE0OyIcTd9smi7c65GBOqa7eoANDn57ti9MhaOiqoFXyk5rdAqs4l3SUpl8v5zMOzoAsrBLh3PVeNUVtHUeTBQuBwoikz8k8z6xqNht/eqlar+TAiFx0xxHiUSarFLazZraYt/1uzdx7DqYkd0owh/1sFD9u0Yb4QbpDkdoSOKWm/kjSvPjctMq70U1JB7ZyLFW+ZR5rVSUZVwRbCXNRaVIEYyrPQcSZ+tLe3t1DfngVdWGCwWCwil8t5Ztcy1cQGuDkEmUcLQxBIVLdCt65ikhA3n2BocV7GGgBfIksndBLTqObVsBiPJZH66tYMnRf20pJawH4ORKFQiC2LZp80TBkCJPV+6k7o+Zr/YC0ojXAQ21HBrNYREM++PIoymYyvJKT3I6lQVZfDgpnaDxXKTB9Pp9N+npzlUuILawmwXLauCuROMZPJxK8BIEPSzFd0mFpIq80wv5xpyZ1OB865havIEDQK+cVqIitDhv6HyAKGlilUuwKHVklIUHC1pIa7rK+vgGoI05gXcbD+fhJZoI7hupBLcRxXbG9vz4cGrWvIfmvJMj2u48Q/i69EUeSTjZIA39OiC2kJENlnHJnaN5vN+gIPLA2tfrwWlhiPx34hieYAUBvncjmUSiU/iXZ2dhbSRJzMJJahUvRZ3QF7Lf9bv5zPraboUWHLENpPK2g0GuHNN9/0RVS4JZdF+fVZ1HKx901iUIstLCIQ7HV87kWjA1z/sbu76wV/oVDw0Miv7AAAIABJREFU5jzPUYtHaxloX60FwWclbnCcfj0rupCWQKlUQrVajeWVA/svibvHRFGE7e1tb27qxB2Px9je3saDBw/8fvbT6X6hCKaZ7uzs+Dg1VyouSmquc1mrWgCq9YGwBlKG174nAYIhPMC2SUALAB4/fgzn9vdb2N3djSXkUFhZQNMysDJFEs2Ln89zk0g2orAora+ve7CY1h7vqci/Rme0TzbEybEgJsACM0elUZ8GXUhLYGVlBel0OrbvIHCo+SqVCmq1GnZ2dnyOOI9H0X6CULfbxcrKCmq1GqbTqd+2iuFBbsjB7cWOMwGtNuMyZU4q9bGt9icpUMjzLIOHwET9PdQvAnCXL19GJpNBr9eL7apkIwzzyJrBeo0KE2syh4SKtZKAuADhblGLEMt/JeEhmh+QZN0o0XLkH92IdDqN4XDo3YLjKIqTpAspBOr1upfKTM6xcfdLly5hd3cXu7u7fkIQeBqPx94E1tRVkkYUuLvx02wyYesCaNSAk17TgJWS/H5+1jbs9da81olfLpdx8+ZNVKtVvPnmm7FJTKawzGozBG0/NbSnVk5SJCMUrgtRKpVaqI4DaTwe+9i9Bf3s/oH6jApqKilYqRWs+WczTE+bLpwQcM7h0qVLaDab6HQ6vmKuat18Po9isYirV6/i3r17qFarKJfL/qVpuSklmycOHDJqJpNBuVw+VjzYugUkTSYKTTqd7EnAWpJVoPdgHyzewXveu3cP9+/fj/WH/bW+vLonKrg4PnQneL6tOajtWc2r97XWkR3LRYjMqOtGrDBQgWxBThWeHDcuIefislwu53exjqL9moa65Pw06cIJAQJt3EFY0VvgcCFOsVj0FsP9+/dx/fp1n+1nM7wU0ed3a2J3Oh1Uq9WFhYBO9BCar3kE1KCK1APhoiRJ99KQG0mf0S42ojtFJrBYg4b11NrQ8zSbT6sohTS2XVnItjQ6kAQcHhcPiKIIzWbT70uh/VaQ2I4L72XDnrpxLatO2TL35XIZOzs7C/fxJOnCCQGistyElGFABb0GgwG63S5yuRxqtRoymQy2trbQaDRiZqrNdAvVBSRZfzWJbJZZyKzmZzKkvZ9OQJvWq0CZ9bGTBAAwWwD1uJmP7JcKIo4V+znPXFeMgN91XELPz+d8EtrZ2fFzhf+1Pfbd5iFYTIR5KNzqjCXrrNI4yz0JL5wQqFarKBaLPvxHU40lwjjBB4MBBoMBHj58iFqthrW1NW+uMVwExMEsm0ILxMHGRXagVSSa1yeFAq3mDyHxSdrRAocKIIaIz2HdCxsRUGKb6kKQYe25FAY2agEcrrXXrDx7H31m9kv/E79Z1O/u9/totVqxbE/NSrTgLJ9J3QO6mnQBCAT2+/2ZZd/LegKnSLVaLZaWS6SX1YPISEz84WTo9/t+B2N9icqIKgSoPQg0snbdSZJmqFkNScsjBJhZiwKYFRbWUrAxcCXLAJb0d8UY9F4KnqlVoBaPjRJYJtT27PHjjj3fcxRFfkdi7kwNIGb16fgRKNZ8AuagdLtdn3tAgcR+c0v6J7GwnpYupBAAZte0M/xFmk6nKJVK3qRrtVrY2tpCoVDwm4tovUDNLmSikC5BrVar2NraOrJ/nLBWyyuFwmFqis4jy5BJvrp+54TWwhih+L3157V/1uRX5tbn1tWE9vzQf+2z7Yu1ap4kDMcl57qeZHV11VuQvJeCmVQQFAjpdBo7OztoNps+K5XCmwKCkYezEAILiUfnXMM597vOubedc99zzv2kc27VOff/Oud+cPB/5eBc55z7n51z7zjn/tw599ln+wiLUzqdRr1eTwSPFGRSczKfz2N9fR2bm5sAgK2tLUynU5TLZW/iAfBg4srKil+bQEGwurq6UH44axIeJQD0mdS3DjHhUZGAUH6BvZcVMoui7fMQeo6zdZ1sX/jflvLWc5OSiujCPInJzaiG5gVoVKndbvsw8t7eXqxqMPszGAxw584d7OzsoFqtYn19PbaFmfb7qFoTz4oWtZF+FcC/iKLoNQBvAPgegF8G8PtRFL0C4PcPvgPA3wTwysHfVwD82on2+CmIuwGHtAkQn+hEy4FDvzeTyWBtbc3XD+ACIa07z3x6tkfJvuhyUfW5FWRKYjr159UMBmaZSENbSfe2wKFaBhbrIM0D9EJMbcki7vPI7i1w1P0BxEzu4xBDepVKBdVqFaVSCblczueW0GohbqBW3GAwwKNHj3D79m2MRiOsr6+j0WjEMg9pfTIDkVbqadOR7oBzrg7g3wLwdwEgiqIhgKFz7ksA/vrBab8B4A8B/BKALwH4zWh/5P/1gRVxJYqieyfe+2MS68PrzkBKFmFW01lxBG4+0uv1MBgMvImXTqd9jXqCWLQYPvzww4W1J8+bl/xiQ5TWrw61mcQsKkD0+YH5acQWoLRC1fYz1G8bHuQ9KYT1GM9XC0JBRy2GolYd3+UiwKx9RoYJNU2YYT9aerTaisUixuMxtra2/FqRlZUVVCoVb/IrAK3WJ/NTzoIWwQRuAXgE4H93zr0B4FsA/iGADWHs+wA2Dj5fA/CBXP/hwW8xIeCc+wr2LYVTI/p1SWh5EqilnznhuGCGhUkUJVciwHT37t2F+siaBJY0AmDDbJrAotrfhq+Sogz2HvNW7llLQN0FMmpSPkPIbydpf5NwAN6HjGjBSmV4FRQKAh+HUqmUT+nV8WFIGdiPWNC6bLfbePz4MUajkbcObX1JKgi1sNg+a1ycNi0yKhkAnwXwD6Io+oZz7ldxaPoDAKIoipxzx1oFEUXRVwF8FQCOe+2TklYHsmEtZSRFw8lAtvY+cMgs+p1t8fNoNML29vbCu8zQL7Ra02pdkian2OfRZBydiNbXVwYMWQA2CjEvnKgug8UoQudaoaHPCRwmQlkNb9vhsyjAaN0Lrvxb1CLjBqPU0poJyvwBlpNvt9u+yOza2prHi7TPoSXHWoHoKLfmWdEimMCHAD6MougbB99/F/tC4YFz7goAHPx/eHD8DoAbcv31g9/OnHSy6IQJ5YLr5LQgnUW0Scpseuw4WYKlUmnuJCVjJaHwqv14f7VWklwJNZ1t6jPvCyy+6o0a245v6DxtT+sl2mQmNZ9VGOjv6vZYt4Cl4hcl5o/Q/2e9gnK5HHMJ+/2+zwVpNBqoVCozeBKfQ4W0XZ140iHkRenIu0ZRdB/AB865Vw9++lkAbwH4OoBfOPjtFwB87eDz1wH8pwdRgp8AsHce8AAgvsgDiGe/qVay5ijPVebXianLe9VHJEC0qISnxkm6L7+TSW3GI/ujZjT/5mECeq4+I++n54SAOe2bnqvmc+gcvY/9rqFIG7E5ajxDYCSvP85OP/T9WWmKtScpFID9JCSWlqvVaigWi7E1ERwHzUjlM+nuRYuAos+KFnWS/gGAf+qcywF4F8Dfw74A+WfOuS8DeB/Azx2c+3sA/haAdwB0D849F6TMD4Tr3lk/k0ICmE2ysW6Dmq68vtlsLtw/9T/VvLfmt4KA2mfLUEdNLBUw9r8+pzXRk0gZlX0KRRYsIEjBzN/s89nnVtPaald9X1agAftjfJx6ftTyWu0pl8thdXXVMy8FPt0NZf6QC8X+2FJwZ+UOLCQEoij6DoDPBw79bODcCMAvPmW/TpWsz8bfQudpMgsQD28xZ4AT+bjFRJhVaCev1dCh78qA7KMyhyW9XguC2ONKNK+T3AFqPE54i+wreDgP/OM51tfX4xqRsMxlXR4dg+OCb7rjdBRF3oQn42uNSRV8IVyE2t9iSnbunTZdqMpCIQYjWUQbiJuSqtFCRJO83+97c/C42Wmrq6tBl8OSouDqv1uNqxVwrLZJAvX0eUjWfbJkXSW9t2XgJIvDChfL9Np3tZBUwFGYqTVi71MqlY6NCzjnYjtD2fReix0pPkFSa0DzL/TZzqqewIUSArqLLTDL7CQyfAgA5O96PIr2c8W73W5MaxznpRJY0nskUYixdMIrYyQBTnqNgnGhayyQqhOc/7lOIsT4pJBlo6SRG4s96PNZ4WxdAn0eK5xSqdSxkoa49FdLjak1Y5+VWI2Nvlh3zoZ6zwoUBC7Y2gGdREkmYwj9VunO73yxo9HIF94kJkDtfBwTj4VKSHbCWM1oC14QRLOhMTVT9XlCGYDOOb+ohfFrvV7bCWlq4HClnz1P29G2QoxNP5sC1frOSbgF0X/m+Os1eu1x0nNV+3PcdXxtmJimvVZJCs214wCdz5ounBBQLWcTWRRUUtKwFCV9t9v1aDG1Kc9TbbEo1Wq1IKps+6EMRmbRSWknlDXVrUnK35WskFQGYq7FdDpFrVbDpUuX8N57782NAJD0PtYi03tak1n7FfKfVQBkMhkvBOx75v3K5TIePXoU7KMldbc4B+z7DYGy1spiv3mdav5FgddnRRdKCGg4EIin23JiKwps691FUYRut+uTf7goxZ6niPGitLa2FmNYZVYlmxB0lAAAwkAbz1dBYN0J9a/VxNakK+7KrMVUFbCzfeb9LVjIcyyQBsRr9CnD6HMpc2rfLYg4nU59nv+iQprvQXEhBYmtENAlz9ZqsCDmUQlVp0EXSgiEiPFbfRlkLsvE7XYbW1tbyOVyPiEkxFCcKIvGfp1z2NjYSATNSLyHFQS8JmR2qvVjmY73DpnX+jl0TiaTwe7uLh4/foxiseir5nLiqxkdotC4af9DPjRJtataAXZc7B+ZuVAoxHafOoqY60Ehos+gewpSCDFSZC2c0Pjq8y4LjZ4icfJoRpgCUnbJKrC/Zfj29jZKpZIXALpfofqNWntvEWKbrVYrxkRAONxlJ5T1r+2zAkcv/dUJHgL+rMm6u7vrq+18//vf96XZVbgkPYd1C2zfFX3Xz1aL2sVE2l/2w5YHJxUKhYWFACsEWcUwmUz8YiAVTLQQee9QNEBzIZKsvtOiCyUEFPXnfgPzBABf3Gg0QrPZRC6X8/UI+MJsbjuwv+acWWaL0Pr6eqzNeQxrLY6jnjdUDkuP8Xer4fj8+l01e6lUgnMO/X7fl05TC0pzFZKEYciXtvfU39XPVldFhYAVNLyHjbhUq9WFk4YsLsB2qf3V9eA84LvXVGV9b+pWsX9ntTPxhRICatryBfB33d6apH4vsL8UWXfnJdHc5j4G3H9wUUugWq3GtqKywJ1SyDI46rgygp7HZ7TEc7UPHDOODwXi5cuX8YMf/MCnO6vfbJF/S+qmJJUct1EF1uqzz6oMai2JUB+OEybku7V4knMupvFJvBcVDasHaaKQzqFQluRp0oUUAhaAUkmuRCAQ2N+1yE4AnWDD4dDvTMTNOhfdCntzc9MnoyiFMICQj6lMYjWiEiczPwPxEJdaMxZ0tMKRdfM/+OCD2JZaShyjUIqz/a5gqu4FYSMC7JNlPItjqKtgnwuAr/+wCOPZ6IBGgXRnIrUSCFRqLUGtMGxDovYZTpMupBBQKazAERCfpPTtiQGENCyw7zPu7u76fQmpMRbZaLJQKGB1ddXv7WfNViC8f6CS/n6UK8HzrVlrfdIk4NDmDeh4hTQdGZuMbAVASMDpfS1jhASdPYfovD63nsv3zSpBR5E+hzK8xRosfsK+WKFN64DjGRLsp0kXKmNQfTt9YRr35QvkeQwnhfz1KNovGLK7u4tOp+MjDVpZ6Cja3NxceC35IufoBLRLpJNclJDg0Hi9nhPSeNom/XUL4ml/2HZI89k+antJ57B/VohZH1zdk+l0uvCKwvF47O9pQVl1ZyzQp32huzmdTmNbnmt4egkMngJZTQTEGYUvmebbPEBrOp2i1+thZ2fHuwxHmeMhunXrFjqdTkwrUjNYRN766Uq8NoRVWLLts+9JoaykdtTUViuLvjuvVVKgUvuhwsaOHc9RLMC6AooH2DY0BKzPXS6Xsb29PfNcIdJ+UnFwLFVA6ljp+DKSwJ2vGEZUWtR9PGm6UEKABSCAQybVF0HpzONqJZD4ovv9Ph4/foxutzsThz9OaevNzU1sbW3NaE0rTOzEJ1l0P9QGP6u2OSqEqQyedB4jDAq2KrNY3ELbolZUAWLvo4yiz5/kO4d+twJB+1AulxOf3z6n3l/3CwDgC46oS6mfp9OpL0w7GAxieQS61mKJCZwCcaJpBWAgDphphID+Pc+h9CcG0O12ZwBCmn2LhAfz+TxKpZJfqaYTwZrLKgxC2lpJrQI1pRWos2BWEkCmjMmx43Zatn/zhEoIbwhFBvi7Amu2H2ohWUtDLRIgbFnYkPBRApuCXWsUDgaDWPHRUFIUwcBer4d2uw3n9vef0Pvr88zDc54lXSghwJi/tQJCVoGGEC1Cz4mlRSQmkwlyuVzQrE6iQqGAx48fB0OOlrTNUHQACK/Nt+2Grgt9t0yrwkN9fnv/ENCoJnOSS0YMxlYSCj2T7fM8fMCCl3oON4lZpPALGd0KFwso65hwZWm/30exWEShUPC/h+bJUgicEnHS2JVk+vIo+RmuokDQFYKMGPR6PaRSKb/clOHCRajRaGAwGMzEyFVTJiHgejw0wa3PHwqT6TE7DnZsrNYPhd+sYFA/nIIgJLB4jq5AVNBRgUQr1ELPrmOjUQr+ru0suqLQanr2kYKBwo1CjMli4/HYlxwH9i2IbDYbBKEXnTcnTRdOCDjnfJ45kBxm4ktmuIlmXa/X88cYOeDmptxGatGXubKy4s+1oFVIa9sJbEEp4HDyc3ImIf/A/PJjGrqylhAnuu0fhZlF4ZVUM+t5vCd/4z1s/Uf7rNoumVBxHRsa5Lk8vmjSEN8Hn5GWQUhoTqdTLwAKhYJnemaXptNpvwCNVsFZ0oUTAuoL86Xa3WS0AqytOGuFB008MiStg6PIOYdKpRJ0G0LMa7Ugz+PvSbHmkAblOCRpcTtWoRCWPT+kpfm7lhlThuaYEy1Pwic06gDM3xfBujEqBCyKP50uviGJCj61hKybqP0vFAqxuRTavyCEr5w2XSghwLBeuVz2DN7v92dqAWYyGb+hqObX06RThuBE6Ha7GI1GyOfzC1kC+Xwe+Xx+ZgLxs50UiySX6HV6LAT6kTkp1EL1FdXk1zbVLw5FI6y7ocxtmYZ/o9FoRrOzT+oiaGhO340dQ0t2bPiZuf1H7QuhoDJ3H+LvtpiLAonWogIOQ5Zq9TDP5CzowgmB4XCIUqnkfTYyLDeS0LXg9uWxDVJo0i2aJFStVoMael6EQLECew01mwJqFgOwboBF3BXcCrVJMzgkpJJIta+m0CpDqrVihRHfgS7U0lz7kBCy4F1ISCgD5vP5I4VAr9dDLpfDZDLxkQGOn6L8xI+oQBRI1WXr+qwEC8+KLpQQAIBWq+U3FOXLJKhnE1kIAqpgAA6LQmpOOH32RVeC1Wq1ICCXxGAWI7CkLo5F55PaClkT1m/WSADHJAlxZz+UCZO0m3UvQgAftT1wyGgUBszDZ76BtSJUuKhQC41FsVhcaMdo3p+VpQD41ZMaelYBwGfTTFLr7oVCnadJF04I6BJi1vVjqE9JkV4yC7eb4q60URQhn8/H/L1FhQDxABt2DDGDBcRCpKZ4EiDI45aUWbStecCiAplqaYR88NB91L/mtWpaWxeCQlZ/42c7NtrmIrRI+jD7ysKjtGqIFVmQVQXDcDj0LifbsQu3zmoZMXABhQAALwC0NFiIOMGHwyGazSa63S7S6TTq9XqsHj1wuHhmkT0HU6nZnXCsZuXksBGDoyyGo8x0ZQz1S9XVsECcfubkDglN/azuiAUnrZWSFEGwloCNThCI1ftzzNSNsP202ATzO+ZpY3XzKKSsn89+sq3BYIB+vz9j8dkIyVlhAaQLKQSotZPqz3MSTyYTdLtdHxbk/vLT6dSbg0SYGede5IUWi0UUi8WFTfwQwycxu0Wo7e/8rJrftqmAHZ9RqxDTLVCQzt6HTEb02/ZLGSEEdqowUovBZufpGPEarRitzJ2UxKWFUJKIz5xOp2fKiKlPr4qBz8RaC0xUo3uguQxLS+AUSc18IA4QAYfakduYT6dTv78cSePQnMiM/S4KCuq92S9lCotkW0GgiDnbmucGqGYOmf7qV1tgkO1aJlGXwFIU7Se/FAqFIOhl+5DNZmOmfWhlpb43/W/vr2Nm+xTqJ3GheULAgqRahIbPp/0g4GjzHYA47hLKuThtunBCoNPpxHxYfRFc580Jr6Wj1CekL8jJC4TTYZOIm4yoFlQzUZkrqU3VIoo+W62p7fI6Prsykk5MtVDUhAcQc3ey2SxWV1extbXlz1F/XLGDJFLGpp+ttf0tYyaFHxVDsP0ORRDs91KpdOR6D2aG0hogEVRWwWOFM61EdQMGgwFyuZxPRDsrunBCgJNsMBhgOBwin88jlUr5CUBgqlAo+EnMyc28AjUL2aauQJxHzu0nCVlXIIS42wnO8/ifGoSMqeG3JGZRE98ultJ+WKKgYXHOyWSCjY0NXLt2Dbu7u7FtuuZR0jOyv/l8PrY2X0OSymBJAlKFnLoF+uyhaxatLaBhUs6hWq02kwikfbWl7Eici8vowCkTGYUJH5lMxjMR04k1tstrtOy0CghiBsqM8yidTvssNQuW8f9RmksZ2bokVotbQaIlrizpRFQBxfHIZrP+2Z1zuHPnDu7fv4+VlRW/hoLPqJrSLsyymXaa6qzfad3YUmHzyI5n6LlC2ECxWDwSHKS1p65SOp1Gs9n0xVZDhV3VxVO3gi4Pd7A6K7pwQqDX63nwhplirAmoIS8yNc/V5BDGivf29rwAYLThKMrn8zEGOcoXtJNSzW0NB6p2TLICgPhuSiH0P+QXc7LTdx6NRrFQ2Ztvvon19XW/Pl/7YqMvKqisFaJMEloXEPoeYmgVLiFBaAXBdDr1/vs8a07DldPpfkIQU4Np8usqSFojxBAIlGqE5SyxANKFEwKUvBrnpU9HzcM/55w3T0nMNGRFIZXqiySccMGKZW5GF/iZRJ+fpNpKzWYb4iOpH2qTcyyCrn/KOLqElhOaLlC3251Za8/rQua6tmvHQBnQCin+D7WZVDB0HqZC0jHI5/NzhYBuNsv7ZjIZn0moYUKCywwtWnyJ1581HgBcQCFAv55ai5uAaiacSurRaOSlOMtD9ft9rwUo7bPZrA8bzqNqtZq4ei8UFeBnBf8saSiORCZT01rbspgAx4DX0lqwEQP2nT705uYmfvjDH/rdhyyFzGurmUPhPHUPksA8O3b2fvNMeysQAcwIshCpBULBagUv3bThcBhzjRS0zWaz/vNZhgeBC1ZoFDhEsG2JKOAwv90ivcD+OnBqei4PVWZhLYF55JxDrVZLzNqbR8rItr90QzShxf63ABzvqVaFMrougNHsN6vxOp2OX5Cl7bIdvZ+2Za0bi4NQuGqIct64WIGSRLZPagUdBQ5S0Nn33O/34ZzzFiX7VCgUkM/nPfZEbIk1KphvsIjyeJZ04SwBYJ+hV1ZWZoAm1bwaKmPhDyYFMQU0k8l4E5G1BucRN+wADnPg7b1DPmwI7FKiIFIGBeLa0UYjlCwjEQHX+1PAMAOOFhQBQatZ6TbYEKsmBukzKZYROs/iB0nuD38LRQcs2bE+qrbA6uoqOp2Of+d8RmB/n8pyuRzrF+9Bq4BblvEZKDTOEhQELqAlAOy7AIPBwL9Qq7kUydVQlZrWxAZsuHAelcvlGYvBCgLVkoqQ83uIFL3XScgJpyEqpRBaHbKEFOhi2yQKmFDlHY6ZPpdaKApoanIQ728X4WiyjvZDXQieF8Jc9FxLUbRfbmxeTgNxG44ngcBSqYR8Pu9XpVqhRMFMi4BhQ4YZz3IFIXBBLQFNsuGfpoMCh9qP2pWRAgXEhsOhjzEfBUABhysHF0ktVg1nJz2fATjMVdeMR52kfBbL1IpiayjLhrj4mUxCECyU6aZhPI6nJj3pvUJW2DysxD4/r1FBon2wjKX5HqHrSdlsNnGj0p2dHf88rCcwGAyQz+d9ZiTXlOjKU7tWQcdhkfqGz5oWsgScc/+Fc+67zrm/cM79n865gnPulnPuG865d5xzv+2cyx2cmz/4/s7B8ZvP8gGehPhiLNl4Mn1g6+/yd6YWA1hoc0tWIQLCTB4ixR2oGa3GVrI5Dqol9Vk0EqCZhhodsbURdN89CgD90/uk02nPKDp+aulY0FGFMr8n+fo2/Kftaf6EBXvtWFMwaIgviarVaiyfgGFlLksnRkMhpGtJFINSUPmsQUFgASHgnLsG4D8D8Pkoin4EQBrAzwP4JwB+JYqilwHsAPjywSVfBrBz8PuvHJx3rojIszU3lUJAlWpZTpx0Oh1cihyiWq0WRODttRZ8siCl9k/9fQomJp8o41uAMEnI8TxtV4E5NbXtYiD2WwtnWNRcrREg7NNbsgJN3xnvrct0rRuT1L4+E3BYUSqJaLqroCRD65iyjwRtuWRd55xiBWdNi7oDGQBF59wIQAnAPQA/A+A/Ojj+GwD+WwC/BuBLB58B4HcB/C/OORcdhZqdEjnn8MILL/g68CSd8MCsNrFtUCPzpR6F8GazWZTLZZ9eq4knOrlDfre9t/1uXQBtw6Lz9nrLqHx2+539tUwVAlYZLdH2rUbXEOBRFOrfvAgL3499n0nPqrjGvAgBE6SYLKVmv747K+hIdou6QqGAXC535sDgkUIgiqI7zrn/AcBtAD0A/w+AbwHYjaKIjteHAK4dfL4G4IODa8fOuT0AawC2TrjvT0xaZtqCXPyvk9cCddYSSKVSR75I1izUHHsLjFlXQS0ABbb0dzXdKVwU1EzSuCHzWhnVfk4yzxVcZB81/MXvIbdF+xISbsrI+hxWeKqVMy9MG8IwmLAznU594k/I8uJzsIhMFEX+fKYLazYkrRN1DdQNootyHmgRd2AF+9r9FoCrAMoA/sbT3tg59xXn3Dedc9982raOS7u7uwBmN+zU//Y3G9ZSIC0UO7bUaDRm2rekAifp3CTrRbMctX/qY6sFY01l/c7zOcl1TYU+f6gfajWodrSltbSfllRTB4sRAAARZ0lEQVTo2me25r6Ou3V7bNtWoOpzs+157gMz+6z7EyIVYup66WrC80KLiKJ/G8APoyh6FEXRCMA/B/AFAA3nHC2J6wDuHHy+A+AGABwcrwN4bBuNouirURR9Poqizz/lMxyLbEnxEBMA4eQcS2SURUI81Wo18TxlCJ007CP/W9NWJ2AIOFSATZN/KLz0Gr0X29NzLQah56sFYJmC99Dz9LlDwF+oDf2cBNjSMmNf9Ji9R+jYdDqdWdth+8H8iHltkSzmZK3KRfCQ06BFhMBtAD/hnCu5/R7/LIC3APwrAH/74JxfAPC1g89fP/iOg+N/cF7wAACxDDtg9sWFXooFwngeGSQppERKp9N46aWXYnsOHjUBkkxFTnbFAKy/HtL+1oJQBrS+fgjBt1o7tM7fhv2oWcmcKnCVrPYNPbvmaST1IWRdWPM/BLraPs17L5pDYYvLaNvWbQw922QyOXLunAYdKQSiKPoG9gG+fwPgzYNrvgrglwD8I+fcO9j3+X/94JJfB7B28Ps/AvDLz6DfT0xRFPllwZykeox/OvlDE0dN66PwgGq1GkOQ55E1hUN900lngTk+D6MWmgDFdq31YykkPOy9bH/0OtV4NhrCz8d59pDlE2pHzwnlFiRZBNY9SAIHNYoQclkULA6Fb7V2QNKzngUtFB2IougfA/jH5ud3Afx44Nw+gL/z9F17NhRK7AktVEmKDlipnsvljgzzODe7C0+IQoxvkXDFAGyftV8WF+C11le316n7oONg8wHUegj55qFl1bSc9Dxq1XmgXlJ/k45pm9Z6Szqf/RmNRiiVSh43UmJEQK0bJRslCFlhOmbnxUA+H/DkKZIWvST9/+2dXWwcVxXHf9deJ67tNM5HleZLaaJERBEqNOKhFTwgPkSpELz0gQqJPlTiBYmCkFArnnhEQpQioQoE4gEhQJQKqjxQQWkfKlWGuKDmozF2UzeuExPbWXvXWW/X7l4eds7k7MkdfyStZ9y9f2m1OzN3Z87cmfO/55x77r060GSVNNS3Lb9FoVaLCczNzTExMZHpa+pzWVNYv1QhUrLldSafVlDZtl1n+jzysTMO2SCjrittEUhdSqxEE44tZ+9/JViCCR3TFkjIItCy6TI6rpFFXraeNHmF7iUUYwndbxGsAOhAErCtg+yzH42QeQo3/NTVLAHvPWNjY6l5nlUmyzS35qr+jw0OivyWcGxcQ/v/Osdfl7euhiYJ2addAm3uS8xCWv8QCdjrZcHef8jvt3LoNGr7X3ttqZeenh6Wl5e5evVqpixaHk2cUi/WPRLY555FiHmg40hAr/yTpVwQfkghf3Ot04zPzs5SLpfTmYblGtrtsNH1LFNZv3T6tyi+Hgqs/VMdzNSpx9YN0IoWUkDpirSWif2PtJpyX7YLc6W6DkEHPFeyLERW+Y9GVqxFiHNycnJFy07nc2QFIK0loGWWcwCF6SrsOBIQxbMrweoHZTO/4OaXx44qXAvOnz9Pb2/vTampoiShYFbI7LQtkG7FbcBJFM+mScu5NWzAULsRosyi5NaF0vVkSdGmy0J7YNG6GhYrxS90HYVIW08WY+tWuytbt26lWq1SrVaDMmTJslLQWEO7EFlWX17oOBLQLbF++aw7YNk+ZCl0d3evK/e7Vqvx8ssvp7Mc60BdyLS304dpWXUUWj4ip43S61mIQya4VkBRdtsaalKwvrQlINsTsZqy2l6GLMi1Q0SW1XsQcu8s6UhPikydvhr0/Vhlt/6+dQ20vEWwAqADSSCE0IPUD8sek++urq51zw83OzvLa6+9xrvvvpt2G2o/G7hJ0bRMK7UgtnUU2CG9lkxC3WjW/5d9WkZdPmuCFD07UBYJZd3XWskitC9LOS0pQWtcR71eX9UK8N6n603Ya2SRm7Vw9HOUruq80VEkUCqV0rkFtRkaauXh5qi0lJUWTs8stB5UKhXOnDlDs9lMc9Vtq6Ll0matNm+1aWktGPui6viCvoYN6mWZtvpl1j0I+vqrmfOh3g2pWy3TSsh6Vva3bK8lZiCxgLW0zDIGQOpNx1QsmVnXzGItE9FsBDqKBAYGBti2bRtdXTdm4QlF0bO6fqwZDKy6ak0WyuUyQ0ND1Ov1NFVV5NGBOn2tUM6AyGZbOv1i2ui/dYFC921N92azmQ5Ptll7OngZipHYiLmVz8YUtCumh0KHnkuod8Ke21ocIq8MdFpcXAzmBWRBrB47T6W1Hm3gV8usE47yRjGk2CA0m810jQEZxiljveUj+0ulUjpphFVKaQV1oOtWUK1WGR4eZnFxMZ21J2SiZyW32C7HkFWjzxGKf+jz2SCjvp5t1eQF1z0AcKMLUlsvurwOkFmEWvUshZYyWg5NmrZFDsURJDj89ttvr/k5ahLTPSX6nnV8JaTooQBwniiGPbJBqFQqvPLKKwwMDKQWQG9vL9u3b2dwcLBtWKg8QN2vbh/aHXfccdsPcmFhgVdffZWTJ0+ya9eutkCeRK6tWxIyO0NdVNa9kPuQbd16A22LZ+iyOi8i5CqJXCK77u2wUXF93lDcxbo4oWtotybUa6PPKXVj3SNxxaampta0XoRA6kx+25iIvCcysMzGTrRcec8tKOgoEoCW+b6SCS+zBImFIOwu+eSiKKVSiXK5/L5MCNFoNDh9+jTHjx/n0KFDqalqWxXdssjLppXYRqKzlEmO6W9RWtvqy0fnMISUV1p5WXBDn1f+J9maevJW2dZWlr6Gvo4lk1AsQdeP1KNACG55eZne3l7q9TqXL19e+eEYhEhGtjUx6ZiNJk9dJ+shnw8SHUcCq0GUutFotM3/tlrk+HaxvLzMuXPnmJ+f58SJEykZaYWxsNaJbmE1YegX1CqSPo8NRlpfXCwj+8Jrt0gnPunZjwW6pdcKKrkb+j6s3LbLMTQuQP9PZypqopRemYsXL667Nd6zZw8zMzNpboFOj7buishgsylDLl+eiCRQIHjvmZiYoFwuc++996ZrI8jsNzawlWVmh35nbdtWVr51QEteYm1ah/xzgfXB9cy7uoyetdh2t4WsDVtXQJvrZo9bJZS1I3p6ehgbG7ulmX7Hx8dTKzEUJ9JkI5aP1If3N4Yfy6zVRUAkgQJiYWGBoaEhDh8+zLFjx9IWNdQ7oWHJYSXfO/Qf2bYWgARU7ZiBlYKQNo6hzyfKYoONNtBq78eSkMijW1Xtssh+mQZcgq+XL1/mypUrqzyFMCShyDmXBo+3bNnCwMBAei1ttWjlr1arqcukE8HyRiSBguK9995jbGyMmZkZjh49yp49e9JjNjAV8p/luIV+QUNEEVJWvX6BfMv/rb+ry2gzXLsBWrl14NWSgXULVro3HfjTWZbyHyGASqXC2NjYbSfpiDJLa37t2rXbOl+eKIZTEpGJubk5hoeH065EPTjIugI2z0CQ1WLLsZAVINsSA7DTk8l/tesgM+WIYthFUCxZ6bkIoX0mIiljYxMht8b2aEgrLa6A1MvCwgJnz54tRJZekRAtgU0A7z1TU1PMzs6yf/9+Dh8+TF9fX5vPa2f0tQqnFSgUTJRyNuhmrYVQC6wj8nYlJ00ooViCZN2FiEoH+ux9yLclJfmfEI+Mq5icnLylQGAnIJLAJsLS0hLj4+NMTk5y5MgR9u3bl+Y16CChKJbuV9fQZUPxAh0LECWSFlVfS7eo4h9LK2wV2AYFNRHpORLFp9bya+LQRKTTmbU8y8vL6bTyjUaDiYkJxsfHowWQgUgCmxBLS0uMjIxw8eJF9u/fz6FDh9LhybqltD68bYm1Mss+u2KRVjAx80Ux9SQaenl0MfXF57e5DdBu+ltrYKWMOilnM/Ukp1+7RHNzc1y4cGFNS8R1MiIJbGJoy2BwcJCDBw+ya9eutEvOEoFWyFAWYmiRELjRUnd3d7eVEaXW/fESmJOFOsSKkOM6z0AUWvIDdIqwHetgo+02G1LyKrz3VKtVLl26xPT0dGEi8EVGJIEPAZaWlpienmZ6epr+/n4OHDjA7t276e/vb1tjQff96z56UTZRciEIad17enpuitBrshCFlAE5EheQsQ160Q35b1dXa5JW278eyrsXucVd0DMi6TqYmZnhypUrzMzMROVfByIJfMhw/fp1RkZGGB0dZdu2bdx9993cddddDAwM3NSdJwQg6yHI9One35hws7u7O02zlnwFSwDaIpCuOMnK04ucNBqNNGdfdzuGMggtxMzXbkO9XqdcLjM9Pc3s7Owtj+jsdEQS+JCi2WwyPz/P/Pw8o6Oj3HnnnezcuZMdO3YwODjYls0myq0DcdKCNxoNGo1GOgR7cXGxjQw0kcjoSx1s1MujCRlA+1h62W/32cSlWq3G/Pw81WqVSqXCwsJCYbLuNjMiCXQAms0mc3Nz6Zh5yXATQpDh01pBhQj0vq6urtTPh/bJRUSJhQh05F5nB4pS29iEjuxLvkGj0aBSqVCr1ajVaqklEfH+IpJAB6LRaHDt2rU0y0269/r6+ujr62NgYID+/v6UGJrNJvV6vc0NEKIQi0CsBvHba7UaW7ZsYWFhIR21VyqVqFaraQ9CpVJJU2rr9XraAyEWRMTGwGUN0NhQIZzLX4iINogPXiqV6O3tbZtp2LoC3vs0U1APOALaknNW8/sjPnAM+8ACwNESiAii2Wym8YBarZa3OBEfIOLYgYiIDkckgYiIDkdR3IEFYCRvIW4Bu4G1rVhRLES5Nw5FkvlQaGdRSGAkFLAoOpxzp6PcG4fNKPdmkDm6AxERHY5IAhERHY6ikMAv8hbgFhHl3lhsRrkLL3MhkoUiIiLyQ1EsgYiIiJwQSSAiosOROwk45x50zo0458acc0/kLY/AOXfQOfeSc+68c+6cc+7xZP9O59zfnHOjyfeOZL9zzv00uY/XnXMnc5a/2zn3b+fcqWT7sHNuKJHvD865Lcn+rcn2WHL8nhxlHnTOPeucu+Cce8M598BmqG/n3HeSd+Ssc+53zrnezVDfglxJwDnXDfwM+CJwAnjEOXciT5kUloHveu9PAPcD30xkewJ40Xt/DHgx2YbWPRxLPt8Antl4kdvwOPCG2v4h8JT3/ihQBh5L9j8GlJP9TyXl8sLTwF+998eBj9GSv9D17ZzbD3wL+IT3/qNAN/BVNkd9tyATN+TxAR4AXlDbTwJP5inTCrL+Bfg8rczGvcm+vbQSnQB+DjyiyqflcpD1AC2F+QxwCnC0stZKtt6BF4AHkt+lpJzLQebtwFv22kWvb2A/MAHsTOrvFPCFote3/uTtDkgFCt5J9hUKicl2HzAE7PHeyxpWU4AsDVSke/kJ8D1AZuDYBcx572Vcr5YtlTs5Pp+U32gcBqaBXyduzC+dc/0UvL6995PAj4BLwBVa9TdM8es7Rd4kUHg45waAPwHf9t63rWDpW3ReqD5W59yXgKve++G8ZVknSsBJ4Bnv/X3AdW6Y/kBh63sH8BVaJLYP6AcezFWodSJvEpgEDqrtA8m+QsA510OLAH7rvX8u2f0/59ze5Phe4Gqyvyj38kngy865ceD3tFyCp4FB55yMFdGypXInx7cDsxspcIJ3gHe890PJ9rO0SKHo9f054C3v/bT3fgl4jtYzKHp9p8ibBP4FHEsiqVtoBVSez1kmoBV9Bn4FvOG9/7E69DzwaPL7UVqxAtn/9SRqfT8wr8zYDYP3/knv/QHv/T206vMf3vuvAS8BD2fILffzcFJ+w1tb7/0UMOGc+0iy67PAeQpe37TcgPudc33JOyNyF7q+25BnQCK594eA/wJvAt/PWx4l16domZ6vA/9JPg/R8t9eBEaBvwM7k/KOVk/Hm8AZWtHivO/h08Cp5PcR4J/AGPBHYGuyvzfZHkuOH8lR3o8Dp5M6/zOwYzPUN/AD4AJwFvgNsHUz1Ld8YtpwRESHI293ICIiImdEEoiI6HBEEoiI6HBEEoiI6HBEEoiI6HBEEoiI6HBEEoiI6HD8Hwtza4T06cdnAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "bVYmLWDDhFrw",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 300
},
"outputId": "652e77bf-0bb4-4f86-b5ce-e60e2512d150"
},
"source": [
"predictions_single = probability_model.predict(image)\n",
"\n",
"plot_value_array(1, predictions_single[0], test_labels)\n",
"_ = plt.xticks(range(10), class_names, rotation=45)"
],
"execution_count": 122,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAEbCAYAAADkhF5OAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAdtklEQVR4nO3deZxddXnH8c8zCSEBDVvARBDCEsImIkgRWU0IWyJBSFiEgmIiWyoQlKQomwIGWtoKoiAQgVhqJCgoKiIWFBSrgKJU3HApaougVVpQFHj6x/Nc5uR2EmbmnnN/w+T7fr3ymnvv3MzvnHPP+Z7fds41d0dERLqvp/QCiIisqhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlLIyIG8edy4cT5x4sSGFkVEZHi6//77n3D39dtfH1AAT5w4kfvuu6++pRIRWQWY2S/6el1dECIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIAbNn48mDX7b/z40mspIoOhAG7YY48NjzJEpH4KYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVEClEAi4gUogAWESlEASwiUogCWESkkJGlF0BkOFm4cGHjZSxatKjxMqQ7VAMWESlEASwiUogCWESkEAWwiEghGoQbxjQgJDK0qQYsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoUogEVECunaNDRNiRIRWZ5qwCIihSiARUQKUQCLiBSiABYRKUQBLCJSiAJYRKQQBbCISCEKYBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlKIAlhEpBAFsIhIIQpgEZFCFMAiIoV07RsxRGT40jfeDI4CWBqhA1LkxakLQkSkEAWwiEghCmARkUIUwCIihSiARUQKUQCLiBSiABYRKUTzgGXY0RxkealQDVhEpBAFsIhIIQpgEZFCVok+YPUJishQtEoEsMiqQBWNlx5z9/6/2exx4BfNLc5yxgFPdKkslT00ylfZKnu4lr2Ju6/f/uKAAribzOw+d3+dyl51ylfZKntVKLtKg3AiIoUogEVEChnKAfxRlb3Kla+yVfaqUPYLhmwfsIjIcDeUa8AiIsOaAlhEpBAF8CCYmZVeBukOM3uZPm8BMLOe/Fnb/qAAHiAzM8+OczPbz8w2LLEMw7GsLG/dyuPJ3Sy7j2WZBCwBduhSeUPuytTq529mo7pRzlBkZusAY/NpbfvDsAng1gdoZmPMbI2myqmE7xuBs4H/aaqsvrSdAA4ys7W7VNYUM9umqbKyjB5gipldamYnAAvMbOyL/b+muPuPgZ8BC81s+ybLMrO1gJ3z8bSmt3V/tH3+bwOObNUC6/r7+XMScEyTAV+DPYj98TxgSW0tI3cfNv+Ag4Hbga8Dfw2s2VA5hwK/Bg7K56MKrOt04N+AcV0o61TgW8CmXVq3bwO/A16Vz1fr8rY1oKfyfBFwE7B9g2VuCSwAbgF+0NS+O8hl2xW4DXh5nds4f+4NfAr4JnAYsHrp9V3JMn8Z+D2wZ11/czjVgLcG5gNnARcQAXxU/q6jM1Uf//8zwG+BdwK4+5/NbEQnZQxweXYDrgE+4O5PmNnqDZY1hdiOu7v7z8zstWa2T81lVJu5qxM7+j3AB8xspLv/pc7yXmxZPDxvZusBuPtC4PvAeXXXhFvr7u4/AjYE/gr4JPBMneUMlpntCFwJPAY8VdffdXc3sz2AK4DFwIPAnsDhQ6Um3Mdxfynx2RxtZlvVUkYm+0uamW0BXAiMcPdD87VdgaXAke7+tQ7+9nJ9vsBzwEPA48B3gAfc/dj8/Qh3f66jlXmRZcjnaxLrNsHdd6qz7D7KmgQsJM78zxHN5D8C17v7J+osL0+iT7r7r/L5vwAj3X22me1N1I6+2GmZ/VyuecAuwKPANe7+iJmdC2xLnPgeqKGM6rq/HXgjcfKZRLQAbnD3X5vZOOC33oWDtf3zz9fmAscSFZxv1bUcZnYGUdM/J0+8bwfeDFwNfKqbJ94+lq362RxK3DnyKXe/1cz+DpgAnAYcAvzJ3a8bTDkv2Rpw29np50Szde3sP1vD3e8F/oW469GgVT6EdxFBtD8xMDMJ2BHYzsxuzvc2Gr5mto+ZzQDWAY4GHjSzW1rh22ktvK2sE83sEOBpIhReCSwjunm+Sk23Mq2UdxpR07rezK7MvsZ3ACPM7AHgH4Gf1FHmi8nAOQz42/x5oZnt4e7nEvvaaZ22OsxsbGXddwdmAie5+8eIrqWNgFlmdiYx1tBYK6eqskxzzOys3O+XADcSrcvX1dgP/EPg9Wa2jbs/4+4fBkYAuwO11DAHq7Id3gmcAbwGOM7MrnX3dwO/IfbJdxPZM+iCXnL/6K257w4cSW9f7LuISwwXAtOAXxJN507L2wL4dD6+gOinWz2frwbcTQSUNbjOpwNfAT4E3EX0y70MuCqf99RY1jyiz3eLPn53VO5wW9VY3tHA3fn4QqKpu7jy+1nAxAa37c5EAK6e2/TvgPFEF9MdRB/w7cAe+f6O+t2BzYlwHw2sDdwM3A+8vvKe6UTg3UuDfc8rWL5Tcr2nAN8F5ufrC3Jf33EQf7N1zO4ETAU2zu19IfAeoutlcu7LNwPv7eY6r2CZRwOfbh0HwCjgn4EF+XwrYIOOyii9kh1snP2A72UgfplospAHzbcymPbP1wYUTu1BCmwCXA98ELi1Er6H1Bl8K1meLSvrtwD4AtF66SGmxlxODljVUNa4PPi2zTA6gmhq7U00yb8MvLrDMtq37/Z5QJ5MnNzWAh4hBmcaH4AD3kIMAk3P56MyJL9Yec8jGRZr1FDephm8OxAn99b+NR/YvPK+HrowGMfyA449RM3OiArN5zIoR+Tv5w10X6uE7wHAj4D3En3KryNakQuAbxA1/62JE+7FrTK79a+P/XINotIzvfLaAcA/1FZmN1ewzh2GaBYdXnntNuDD+fgc4DLgDQM9gKsfAnB47hCjiJrnA8BG+bs5RK2lozNgP5ZnfeDlxADAYuDzrXUiTgBj23ecDne6kbmuS4mR/6uBG4gT22rA2h2uT3X7rgWMrXym1wMH5PP350G5fpP7UeXxBcTA3+x8vjExG+H1wIHEiXfDGtf95VnmNcBEova3hDjZTWpyn1rJ8u1HnBiWESfhZeQMH+AEYFoHf3tbouW0eZbzWG7fqZX9fJ383feB7bq87tXPZlei0jOW6HJ8BHhD/m4uUUkY3clx90JZJT7oQWycUcDkfDwR2CAD9rDKezYBrsvHqwH/QJxFB1VjIWpj3wM2y+fTiKbonURN6CFg24bXewfgImDdDOC7yRoS8DaieTi+pp1uJtGdMznLnUVOOwOOz4NxZI3r9i6iqfkdoltjI6I/7XLgfRl4tdTq+7Es84ja9g1Et9Wb8/UTgK8RJ96OAqGvgxV4FdH8viL36y1zm5xc57ZeyTLtSG+3ysuBb+TjqRk6rZPRscDDrWOhn397c6KCMLPy2pbAbsB9+XwB8GdgSj4fS4wDNHpcvchy/w3Rgr6O6A7ZlzgB/wr4MHFy2Ka28kqt6AA2yHrE9JS5GUbfAdYkppn9Z+vDIkaQ7yFrTERNrt+1p7YwejVRu90wn+9F9Ie1+gpnUWkqNrjuO+fBP5kYBLiS6JO6HPj3unZU4ERi7vTxwF+A3VrbBHgrcSLqaKcj+v7+iqhhTSGa/GOJQa4rs5xdiP7Hz3YaeANYrs3zs944nx+ZB+CsfL4BnfbzVYKLmFP9MWJg7RVEl8/Z+ZluRnRJvLIL6z0yP/d/rXzerc9kdO7jPwWuze3T732NCNqHgEtyvzqh8rs59LZU986Qq/Z9d3XOd9tyT8p1HU90P+yTx9+kPAa3oeZKQZEVHcAGGUXUBg4lmsXPAOdUfn8KcWa+lDgztfrwOuqXJQL+MqJmcgXRD3QTlRp3w+s9obUOecAuzQNms9wpZtPBRREsf7LZgAj1tYDjiD7eEZXf/T2dh+/+GWrHEDXdw4Elld/vS6WG1eRByP/vchlFzJbZnd6unfOA/yXHEDosbz1itP+9xAnobmLQ8SJiPvmEfM8iotXWeM23bdneQZzwphLT66h8/pOIweV+n4AypL4NvCmfH00E/Q75fA9iLu0HiRbc6/v6XLq0/j3VsvP4+lzbe84F5jS2DN1e6UFspJOJ0eDJRNP/AmJgqHWw7AZsB7y20w+S6H/6Sj5+E9GXvH0+Pwe4qOmdhagFXkXM5hibgXURNdW428L3LUSNa0GWeVtlu55O1NA6CgSi9fBjYOfKazsQTbxdKq8tJmesNLV929Z9Ir0tnItzH2t1uczMUNqow/Jm5LadQrTcvgjMyN+tnwf3zcQFGOvSnasa209AY4GTiNrv80Q//JeJ0f4rGOBAGHEie77y/LvEoPF38jMeQVQiziH7+0v9qwTvxvT2dd9MdmXm8wuA8xtbhpIboL8bKT/AE/P5KcA/Ef0yuxP9dIOq8faxM/YQHew3tr3+FmqeetXXMhB916OJE8olxNSnk4irhD5Uc7mziS6bLfJAe4jeAbHZeeBMrKGc+cAp+Xhk/lyLOJkuIvqC30r0OXYUeAPYzvOJLpzPZvljiKb2x4nWxoP0MQVvgOXNyL/T6srYELgPuKrynvWIKW+fGGjQ1bANphKVl1fk8xOJ7oDjiJP+RAZ50idmCvyUGMg7O18bRcyAOH1Fy9Stf7nu++Tj0/Jz+ngec2sTrd0vEiflB8nxp0aWpdsr34+NM5q85pzefrktiFrgBsQZ+zTgI8Rk6Ok1lLklOd2HOEMvBT6fz7fO5x1NverHMpyYO8FVwF752hSir/vHRBfLOjWVtTPRFD45n2+QB8u1GQb3UdOgE9GVc37rNXqbfesAZxKzAK6hSwMvRAvj48RUsK2IPr/WvM7X5vbuqLVB9CHeSdb6K/vWHsTNfeZV3tuVmm/b8s0jarxn5b7VOs5OyP1itxrKmAo8y/IzTd7eHsAl/hH9/M8TM3s+msf4drkfXk9UxI4nBrprr3QttyylN0YfG+cAopZ7DNEvuD4xUHEhvf1KaxDN49bMiEGdRXNDb5IH4dvIGRPESeBe4NZ83uhcTOLyy++Rl7kSZ+KjK7+fTGd9vpOI6VRTiNrnhNzx7gRek+9Zi2hRTGsdkDWt25QM950q27xVE55HnPwav5lRhv9riJr9Ynr7OTfJMPpgjWWtQ7ReXp370rlE7XIpcRXho8B5Ta/zCpZtH2Jmx5rEFV7/QVxm3up/n0t9c8oPBH6Sj7cgpp3tW2K9q/tB/mxd5bkkn69GVO6WUekua3x5Sm6MFWygngyG/yGbb/n63sRAzk51fABtr+1O9HsdQ29t5T150HQ093MFy7Anlf6vPBDek49HEE3yT1HPpP/pxEjup4EvAb8gzvYbEYNNl9HgjIM80M8l+ll3qrx+RC5XY1PNVvBZz80QfGGOOFEbvpM4qXc+tzPC/nSiGftLomUxh2jyt7pdXpix0+S/9vUhatwTiKlld+RrS4jpYLV/FsQA7NNEl0/Hg5o1fTatED6UqAnvVfn9DeQ0xK4sT+kN0r6TELWGY4grcC4gaiitA2UW0XnfcY2UqPFeBZxPXIn16gzh04l5qF9o6gAh+lgfp/dKvenEiPj2lffcTudXnO1PXMxQ3cHOJe5nsDXR9XBOBkRz/VzR/3k2MZvkktzmP2gy+NvKn0WM9rdq+8fn9q3OfKh19gFxFeGutN1ikWjiTqsj6PuxDNXjajKwdeX5e+ntgpqbx1tH/d4rWY6p3Qy1/myXSggfRUy9fE8G8sN08UKY4hujbcNsSe+NX3qIJtulRC1qL+L6+csY4DzJysZu/ZxL9HMeTvQnfy8Plm2I+0jcQAPX3xNzYffKx4cQAxX7E5Pgzye6H95EjMI/0MkJgKjpPE/vqPvoyu/OI/r+xhAnn3fTwQUd/VyeMRl452YAbtlgWWtUHp9K9GueQ9Ryj8/X5xCXvu7axf17NtHd1UjQraTc+Xlc3UZMAVuX3v7Py4iW5YQuLEeJAbc+y8x8aeXB7DxWPkaN3W/9Wr5ub5C2jTCeGBDqIfqIfg78Y+X3L8sd5mriBuj7UanNDaCciW1/81TgwMprBxOzH1rdD43c3yEPhHvovfpoNjH6vycx6nwCUfO+kaytdVje9Dy5rJfPqzWxu+idujdkb4I9yHX+J6LWvQuwNF9/V57UPloJ4WO6ccARTf5TiWZ4ty+xnUbvgPL55P0tMoSPJLpEGh1gLrgvVFsABxCtkcnAmNbvKyF8AA22Ale4jIU30Jy2DXI+0QTYuPKeMUR/XWsi93K12X6UMYOY/rI6MaXrA8SVVzdX3vMKotbb0X0O+rkjnEz0xe6Zzw/LEG5dEz+GGvp+23a8R8gZFPQ2uW+hy3fZ6sL+1Jr6dXBlW26YoXwXcTHLucSUu+O7uFxjchm6WvPNsnckppadT/RJtz7/rtX8C6xze7/3fKLffxHR6tmn+t7+ZkkjyzoENtYGxKj0G/P5RbmR+uxmGMjGovfGHpPzQLy9EuB3EU2OEcSA0Ndo/sY6ra/YOaEthA8lRqIbGSHuI4SPyW3c6Pp2eT9qn/o1Jk+6GxG13zPz9eOIWvCwWffKNmgPHiO6mL5J3MSpte/PIVpi65QMnwa3w7j82WpZfyKfn0p0w/QQsx6Kr3vXv4W18hUsnj9/Y2a/JL7w7y/uvsDMLgS+ZGb7en47Qkvr//WjnH2JAY97iIG8E4npa1sToTyDmBlwPfEhvd3df1PHOq5gebYHzjCzZe5+RW6Hs8zsfe5+k5n9hegTrp27fyG/4eGrZvZhYq5ro+tbwDPEYMqfzGw0cQXa7sRc1HWJG4lvQXT3zBhm6w4sdxPxU4jg/TkxhnIpcdI/ycxeQXS5Henu/11oURuRx9T6wM/M7Eh3/4yZ/Q543MxuIE44Mzy+buow4oT9WMFF7m4Am9nq7v5MPn4DsbHudfez8877x+WXMpxpZqsR/aK/WvFfXGE5U4l7R5xG1Ix2JJqmmwK7mNmT7v5LYFp+vc8Id3+yhlVslb/cSSYff9fMvg3sZ2bPu/tHzMyBS8zsVHf/TF3l9yVDeAQxve217v7vTZZXwO+JJvbfE/Op7yAuKnmYmI96LTEd6kJ378o3a3RLfgPM0/l4d2LWx5VECF9LtHh+Q0w/HEFM7/xRmaVtVlbojgM+ZmZvdffPmtlTxEU273D3Z83sWKJVdHfRhaWL3wln8fXpN5OXDhPTrn5CTAS/Nc9W84lBuY+4+1c7KGtnoq/r6/k9Y0cQ32O2NnH55z3AXe7+i07WaSXlj3T3Z/PxfkRzd0k+fydxL4RPuvttubPc4e7/0cSy9LFsLxysw42ZvYyYTvgq4JbKyf464DPuflPJ5WuCmU0nBtouJmr3JwEXe3x32frE7RW3A07t1j42FJjZ/sQA/kziJHwu0Q35ODE4e9hQqIR09Us5zWwBvbe5O8fdf2BmxxMHze0ZwmcAt7n7d2soryebG5OJ+zk8RVzxtTlxAljqNX+Pm5lNI/oZHyQG/yCmvS129xvzPYuJ2slZ7v6FOsuX5ZnZbGJq4WHu/kjp5alTfj/gBcT9Fm4xs1cR83m/7u4n5HvWI7pjNiWOgefc/flSy9yE1nGej48kxo8uMbNZxPjSge5+T7a61wMeHCono650QbS+7NHdLzKzJ4i5h9cRk/FvBByYmV8ueXFd5bY+FHf/YfYBHU70Ez4M3NlA+O5PXMSxhBhc3J+4tHEx8Ne5HT5J3IN1NDEnVBpgZhOIz3su8c0pwy18xxMXDc1x92+Z2Zru/qiZnQQsMbN57v4hd/+tmS0iplYW+5bhppjZa4BFZjbL3Z8iarlPALj7MjN7HrjFzOa4+6dLLmtfGg/gVvhm39Qf3P0ai6/ZvsjMHnP3+81sGdE31Vi/VIbwMuAg4o5Uv63z75vZusRI88zsd9qYaBauToQwwPvMbCZxT4I3D8eBoCHk98TFJjOHW59vah90fLeZ7U0MKj0KLDCz9d39HHf/XcHlbJS7P2hmzwJLzezNxEDbE5XffyrHZC41sy8BTw+lFkBXuiDM7ADijv/Huvvd+dpJRFP9ZHf/t2q/acPLslpTNYHsj7uYmGP5pJn9M/BVd78yf78tcUesO4ZpKEiXZKjMJ25m3xp0vIdo3b2JCKGDiRP946WWsym5/j2tVqyZ3UTcz+KR/PkwcZKCmGL6tLv/scSyrkzjAWxmGxI1w7nu/s1sMowlaieHEIMGbwD+dyidmQYrTzaXEiPyrwSOcvc/tloCZZdOhpOVDDpeT3SD3TEc97nqsWRmG7amqprZlUSX05XEjJexxHzwM4dKn2+72gPYzLYhLm1cms/XIu47+jQx7WwL4rrrG939ajPb1N1/VutCFGZm+xAXfYzPaTGj3f1PpZdLhr/KoOPhw7GV1Ra+84ib6dxPfM/c983scuKCp4PyPaPc/c/llnjleur8Y2a2JfFVJmu2XnP3PxAXPowiLv+dRsxF3Sl/P6zCF8Dd7yAuPb3TzDZQ+ErTzGyCmZ1KTLc6djiGLyx3scnBxL2m5xFX/B1vZru6+8lAj5m1+n4b79bsRG2DcDnV61ZgmbsvztfGZL/LEuJ7op4zs12Iq9IW1lX2UJQXPowCbjOz18VLw685KEPGcB90fIGZbUXMNrohB/F/SnyJwxE5k2qGmb0yj7chfczVUgPOboePE5c+/sHMdgPIvs/NiPvujre4HPdUYt7ibXmGGrbc/Rbifg/PK3ylSe7+R3f/3HAMX4vLp6ueJFrRR2Wt97+J+0z/GTgou/x+3e3lHIyO+4DNbAwxyHY1UQM+nexuIO65ewsxMfyCfP94d/8vDUqJyIvJ2u73ia+xf9jdP5qvjyauqp1KXF5+bw5Kjnb3J1b4B4eYWgbhWqGajycTHeMjiQD+kbs/VL1aRUSkP8xsI+KeHp8lwvYx4hLjf3X3p8zsZOKCmzPc/RvllnRwaumCqIRvj7v/kOjzfZa44GDtfI/CV0QGxOOmWd8kbqh1IHGp9Vzg8zm28iBx460B37RrKKh1FkTl0t8fEyE8muiTWafOckRk+KuMES0kBtPGAf9F3FzoB8T3uB1BfMvHo0UWskONXohhZpPghUAWERmQDOHViGsJNiOmry5095tz2uvj/hK+r3FX74YmIjIYObb0FeByd39/6eWpS61dECIiTcixpYXACDNbo/Ty1EUBLCIvFd8gBuOGDXVBiMhLhg2zb3RRAIuIFKIuCBGRQhTAIiKFKIBFRApRAIuIFKIAFhEpRAEsIlLI/wFJtpe0kf6K1AAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "_zhXc9YFqo2c",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "8643e7fc-2f54-4dd0-9008-20dd02ca72f5"
},
"source": [
"np.argmax(predictions_single[0])"
],
"execution_count": 123,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"2"
]
},
"metadata": {
"tags": []
},
"execution_count": 123
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tNit8BrnpwN0",
"colab_type": "text"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J5-6jVbKq8Ex",
"colab_type": "text"
},
"source": [
"## Iris Flower Classification with Tensorflow"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "6H6GeQkDrOjd",
"colab_type": "text"
},
"source": [
"### Import Packages"
]
},
{
"cell_type": "code",
"metadata": {
"id": "1EZFlJNypCQh",
"colab_type": "code",
"colab": {}
},
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"from IPython.display import clear_output\n",
"import urllib\n",
"\n",
"# hide warnings\n",
"import warnings\n",
"warnings.filterwarnings('ignore')\n",
"\n",
"\n",
"import tensorflow as tf"
],
"execution_count": 124,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "ehbZbkqP-rsg",
"colab_type": "text"
},
"source": [
"### Load the Iris Flower Dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "V90WMOI2-0EF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 103
},
"outputId": "ed7a9307-834e-4406-e2e1-10f3d45d4884"
},
"source": [
"categories = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth', 'species']\n",
"species = ['Setosa', 'Versicolor', 'Virginica']\n",
"\n",
"train_path = tf.keras.utils.get_file('iris_training.csv', \n",
" 'https://storage.googleapis.com/download.tensorflow.org/data/iris_training.csv')\n",
"test_path = tf.keras.utils.get_file('iris_test.csv', \n",
" 'https://storage.googleapis.com/download.tensorflow.org/data/iris_test.csv')\n",
"\n",
"iris_train = pd.read_csv(train_path, names=categories, header=0)\n",
"iris_test = pd.read_csv(test_path, names=categories, header=0)\n",
"\n",
"y_train = iris_train.pop('species')\n",
"y_test = iris_test.pop('species')"
],
"execution_count": 125,
"outputs": [
{
"output_type": "stream",
"text": [
"Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/iris_training.csv\n",
"\r8192/2194 [================================================================================================================] - 0s 0us/step\n",
"Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/iris_test.csv\n",
"8192/573 [============================================================================================================================================================================================================================================================================================================================================================================================================================================] - 0s 0us/step\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wmoZ5vmfBFMH",
"colab_type": "text"
},
"source": [
"### Create Base Feature Columns"
]
},
{
"cell_type": "code",
"metadata": {
"id": "zcYLpO7uBQS8",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 53
},
"outputId": "d65b658b-e92d-4286-caf7-030fa4feb9dd"
},
"source": [
"feature_cols = list()\n",
"\n",
"for key in iris_train.keys():\n",
" feature_cols.append(tf.feature_column.numeric_column(key=key))\n",
"\n",
"print(feature_cols)"
],
"execution_count": 126,
"outputs": [
{
"output_type": "stream",
"text": [
"[NumericColumn(key='SepalLength', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='SepalWidth', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='PetalLength', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None), NumericColumn(key='PetalWidth', shape=(1,), default_value=None, dtype=tf.float32, normalizer_fn=None)]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SCMis3OYAxWi",
"colab_type": "text"
},
"source": [
"### Define a Input Function"
]
},
{
"cell_type": "code",
"metadata": {
"id": "CUQVVu8X81Po",
"colab_type": "code",
"colab": {}
},
"source": [
"def input_fn(features, target, training=True, batch_size=256):\n",
" dataset = tf.data.Dataset.from_tensor_slices((dict(features), target))\n",
"\n",
" if training:\n",
" dataset = dataset.shuffle(1000).repeat()\n",
" \n",
" return dataset.batch(batch_size)"
],
"execution_count": 127,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "xJ41FeGDC892",
"colab_type": "text"
},
"source": [
"### Building a Model\n",
"\n",
"We can use two kinds (most commonly used) of Classifier models in Tensorflow to build a classification model.\n",
"- `DNNClassifier`\n",
"- `LinearClassifier`"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "5OcqMGSPW0Td",
"colab_type": "text"
},
"source": [
"#### `LinearClassifier`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "dyubbq1CXIYB",
"colab_type": "code",
"colab": {}
},
"source": [
"# Build a Linear Classifier\n",
"iris_linear_classifier = tf.estimator.LinearClassifier(feature_columns=feature_cols,\n",
" n_classes=3,\n",
" optimizer=tf.keras.optimizers.Ftrl(\n",
" learning_rate=0.1,\n",
" l1_regularization_strength=0.001\n",
" ))\n",
"\n",
"clear_output()"
],
"execution_count": 128,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "G1VRoGZ3cHuR",
"colab_type": "text"
},
"source": [
"##### Training the Classifier Model with training data "
]
},
{
"cell_type": "code",
"metadata": {
"id": "5ZDS2h5Nbw6K",
"colab_type": "code",
"colab": {}
},
"source": [
"iris_linear_classifier.train(lambda: input_fn(iris_train, y_train, training=True), steps = 5000)\n",
"clear_output()"
],
"execution_count": 129,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "hvTXV882c8Ws",
"colab_type": "text"
},
"source": [
"##### Evaluate the Classifier Model with training data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "2x415iD_c8_l",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "90dc3298-3dda-433f-c32f-3501b984b7ed"
},
"source": [
"result = iris_linear_classifier.evaluate(lambda: input_fn(iris_train, y_train, training=False))\n",
"clear_output()\n",
"print(\"Train Set Accuracy : %.2f \"%(result['accuracy']*100))"
],
"execution_count": 130,
"outputs": [
{
"output_type": "stream",
"text": [
"Train Set Accuracy : 98.33 \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bQVLN6jccZ8L",
"colab_type": "text"
},
"source": [
"##### Evaluate the Classifier Model with test data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "eRC02xFecbnZ",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "f7c1e608-75ae-4e36-cf61-791609040406"
},
"source": [
"result = iris_linear_classifier.evaluate(lambda: input_fn(iris_test, y_test, training=False))\n",
"clear_output()\n",
"print(\"Test Set Accuracy : %.2f \"%(result['accuracy']*100))"
],
"execution_count": 131,
"outputs": [
{
"output_type": "stream",
"text": [
"Test Set Accuracy : 96.67 \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FTNWyYzRcx7O",
"colab_type": "text"
},
"source": [
"##### Use the model for prediction"
]
},
{
"cell_type": "code",
"metadata": {
"id": "e8nJcBsNcvKE",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 133
},
"outputId": "37915c93-f38a-4ca4-ee79-7ec861702dd2"
},
"source": [
"predict = dict()\n",
"categories = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth']\n",
"print('Please type info')\n",
"for category in categories:\n",
" valid = True\n",
" while valid:\n",
" val = input(category + ':')\n",
" if not val.isdigit():\n",
" valid = False\n",
" \n",
" predict[category] = [float(val)]\n",
"\n",
"def input_func(features, batch_size=256):\n",
" return tf.data.Dataset.from_tensor_slices(dict(features)).batch(batch_size)\n",
"\n",
"predictions = iris_linear_classifier.predict(input_fn=lambda: input_func(predict))\n",
"clear_output()\n",
"for pred in predictions:\n",
" id = pred['class_ids'][0]\n",
" prob = pred['probabilities'][id]\n",
"\n",
" print('Prediction is: \"{}\" ({:.2f}%)'.format(species[id], prob * 100))"
],
"execution_count": 133,
"outputs": [
{
"output_type": "stream",
"text": [
"INFO:tensorflow:Calling model_fn.\n",
"INFO:tensorflow:Done calling model_fn.\n",
"INFO:tensorflow:Graph was finalized.\n",
"INFO:tensorflow:Restoring parameters from /tmp/tmpx7p2d6pl/model.ckpt-5000\n",
"INFO:tensorflow:Running local_init_op.\n",
"INFO:tensorflow:Done running local_init_op.\n",
"Prediction is: \"Versicolor\" (95.65%)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vCQkafAODfk_",
"colab_type": "text"
},
"source": [
"#### `DNNClassifier`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "RAEPRpMN96EV",
"colab_type": "code",
"colab": {}
},
"source": [
"# Build a DNN Layers with three hidden layers each having 128, 32 and 8 nodes\n",
"iris_dnn_classifier = tf.estimator.DNNClassifier(feature_columns=feature_cols,\n",
" \n",
" # 3 hidden layers of 128, 32 and 8 nodes\n",
" hidden_units = [128, 32, 8],\n",
" \n",
" # number of classes for number of classifications\n",
" # here, three species of flowers\n",
" n_classes=3,\n",
" \n",
" # adam optimizer for better efficiency\n",
" optimizer=lambda: tf.keras.optimizers.Adam(\n",
" learning_rate=tf.compat.v1.train.exponential_decay(\n",
" learning_rate=0.1,\n",
" global_step=tf.compat.v1.train.get_global_step(),\n",
" decay_steps=10000,\n",
" decay_rate=0.96)))\n",
"\n",
"clear_output()"
],
"execution_count": 134,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "pdEDmhngHmyJ",
"colab_type": "text"
},
"source": [
"##### Training the Classifier Model on training data "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "JLmZwWOqKYBk",
"colab_type": "text"
},
"source": [
"While we train our model we must make sure that we obtain minimum loss.\n",
"\n",
"Here, the third-last line denotes the loss value:\n",
"~~~\n",
"INFO:tensorflow:Loss for final step: 0.047325253.\n",
"~~~\n",
"in\n",
"~~~\n",
"....\n",
"INFO:tensorflow:global_step/sec: 370.872\n",
"INFO:tensorflow:loss = 0.025431797, step = 4800 (0.270 sec)\n",
"INFO:tensorflow:global_step/sec: 393.163\n",
"INFO:tensorflow:loss = 0.027603408, step = 4900 (0.251 sec)\n",
"INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5000...\n",
"INFO:tensorflow:Saving checkpoints for 5000 into /tmp/tmpnx65c0fz/model.ckpt.\n",
"INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5000...\n",
"INFO:tensorflow:Loss for final step: 0.047325253.\n",
"<tensorflow_estimator.python.estimator.canned.dnn.DNNClassifierV2 at 0x7fd0a824afd0>\n",
"~~~\n",
"\n",
"to minimize the loss we can :\n",
"- add/reduce layers\n",
"- add/reduce nodes\n",
"\n",
"but remember always to use optimizer for better efficiency.\n",
"\n",
"If we obtain loss value at final step greater than 1, then we must tweak the hyperparameters (Hyperparameter Tunning) and re-build re-train the model."
]
},
{
"cell_type": "code",
"metadata": {
"id": "30vp6heKGctA",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 1000
},
"outputId": "08d4f44a-4cc9-4b4f-fc3f-42386ff2aed3"
},
"source": [
"iris_dnn_classifier.train(lambda: input_fn(iris_train, y_train, training=True), steps = 5000)"
],
"execution_count": 135,
"outputs": [
{
"output_type": "stream",
"text": [
"INFO:tensorflow:Calling model_fn.\n",
"WARNING:tensorflow:Layer dnn is casting an input tensor from dtype float64 to the layer's dtype of float32, which is new behavior in TensorFlow 2. The layer has dtype float32 because its dtype defaults to floatx.\n",
"\n",
"If you intended to run this layer in float32, you can safely ignore this warning. If in doubt, this warning is likely only an issue if you are porting a TensorFlow 1.X model to TensorFlow 2.\n",
"\n",
"To change all layers to have dtype float64 by default, call `tf.keras.backend.set_floatx('float64')`. To change just this layer, pass dtype='float64' to the layer constructor. If you are the author of this layer, you can disable autocasting by passing autocast=False to the base Layer constructor.\n",
"\n",
"INFO:tensorflow:Done calling model_fn.\n",
"INFO:tensorflow:Create CheckpointSaverHook.\n",
"INFO:tensorflow:Graph was finalized.\n",
"INFO:tensorflow:Running local_init_op.\n",
"INFO:tensorflow:Done running local_init_op.\n",
"INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 0...\n",
"INFO:tensorflow:Saving checkpoints for 0 into /tmp/tmpf039um29/model.ckpt.\n",
"INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 0...\n",
"INFO:tensorflow:loss = 0.9548463, step = 0\n",
"INFO:tensorflow:global_step/sec: 340.898\n",
"INFO:tensorflow:loss = 0.06929995, step = 100 (0.297 sec)\n",
"INFO:tensorflow:global_step/sec: 391.516\n",
"INFO:tensorflow:loss = 0.051905256, step = 200 (0.254 sec)\n",
"INFO:tensorflow:global_step/sec: 413.551\n",
"INFO:tensorflow:loss = 0.032315936, step = 300 (0.243 sec)\n",
"INFO:tensorflow:global_step/sec: 383.534\n",
"INFO:tensorflow:loss = 0.05966128, step = 400 (0.259 sec)\n",
"WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 401 vs previous value: 401. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n",
"INFO:tensorflow:global_step/sec: 404.175\n",
"INFO:tensorflow:loss = 0.03130503, step = 500 (0.249 sec)\n",
"INFO:tensorflow:global_step/sec: 411.51\n",
"INFO:tensorflow:loss = 0.041695025, step = 600 (0.243 sec)\n",
"INFO:tensorflow:global_step/sec: 412.703\n",
"INFO:tensorflow:loss = 0.04021568, step = 700 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 388.643\n",
"INFO:tensorflow:loss = 0.0441709, step = 800 (0.257 sec)\n",
"INFO:tensorflow:global_step/sec: 401.047\n",
"INFO:tensorflow:loss = 0.027175507, step = 900 (0.249 sec)\n",
"INFO:tensorflow:global_step/sec: 412.232\n",
"INFO:tensorflow:loss = 0.029116593, step = 1000 (0.242 sec)\n",
"INFO:tensorflow:global_step/sec: 409.903\n",
"INFO:tensorflow:loss = 0.035351362, step = 1100 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 380.454\n",
"INFO:tensorflow:loss = 0.15880767, step = 1200 (0.261 sec)\n",
"INFO:tensorflow:global_step/sec: 411.295\n",
"INFO:tensorflow:loss = 0.054302238, step = 1300 (0.243 sec)\n",
"INFO:tensorflow:global_step/sec: 386.703\n",
"INFO:tensorflow:loss = 0.03428327, step = 1400 (0.260 sec)\n",
"INFO:tensorflow:global_step/sec: 406.562\n",
"INFO:tensorflow:loss = 0.034377772, step = 1500 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 382.043\n",
"INFO:tensorflow:loss = 0.07531372, step = 1600 (0.263 sec)\n",
"INFO:tensorflow:global_step/sec: 403.809\n",
"INFO:tensorflow:loss = 0.028360832, step = 1700 (0.248 sec)\n",
"INFO:tensorflow:global_step/sec: 403.088\n",
"INFO:tensorflow:loss = 0.035252944, step = 1800 (0.246 sec)\n",
"WARNING:tensorflow:It seems that global step (tf.train.get_global_step) has not been increased. Current value (could be stable): 1894 vs previous value: 1894. You could increase the global step by passing tf.train.get_global_step() to Optimizer.apply_gradients or Optimizer.minimize.\n",
"INFO:tensorflow:global_step/sec: 400.621\n",
"INFO:tensorflow:loss = 0.04120819, step = 1900 (0.251 sec)\n",
"INFO:tensorflow:global_step/sec: 385.629\n",
"INFO:tensorflow:loss = 0.03402465, step = 2000 (0.259 sec)\n",
"INFO:tensorflow:global_step/sec: 406.34\n",
"INFO:tensorflow:loss = 0.025264615, step = 2100 (0.247 sec)\n",
"INFO:tensorflow:global_step/sec: 400.315\n",
"INFO:tensorflow:loss = 0.019488363, step = 2200 (0.250 sec)\n",
"INFO:tensorflow:global_step/sec: 407.913\n",
"INFO:tensorflow:loss = 0.034090046, step = 2300 (0.245 sec)\n",
"INFO:tensorflow:global_step/sec: 383.383\n",
"INFO:tensorflow:loss = 0.025606574, step = 2400 (0.261 sec)\n",
"INFO:tensorflow:global_step/sec: 406.737\n",
"INFO:tensorflow:loss = 0.027330484, step = 2500 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 409.151\n",
"INFO:tensorflow:loss = 0.10570714, step = 2600 (0.246 sec)\n",
"INFO:tensorflow:global_step/sec: 417.941\n",
"INFO:tensorflow:loss = 0.03930735, step = 2700 (0.239 sec)\n",
"INFO:tensorflow:global_step/sec: 384.056\n",
"INFO:tensorflow:loss = 0.026937068, step = 2800 (0.260 sec)\n",
"INFO:tensorflow:global_step/sec: 402.361\n",
"INFO:tensorflow:loss = 0.03000135, step = 2900 (0.249 sec)\n",
"INFO:tensorflow:global_step/sec: 407.095\n",
"INFO:tensorflow:loss = 0.049963575, step = 3000 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 413.01\n",
"INFO:tensorflow:loss = 0.10355372, step = 3100 (0.241 sec)\n",
"INFO:tensorflow:global_step/sec: 384.285\n",
"INFO:tensorflow:loss = 0.02253997, step = 3200 (0.262 sec)\n",
"INFO:tensorflow:global_step/sec: 403.511\n",
"INFO:tensorflow:loss = 0.025050739, step = 3300 (0.248 sec)\n",
"INFO:tensorflow:global_step/sec: 405.741\n",
"INFO:tensorflow:loss = 0.022113536, step = 3400 (0.245 sec)\n",
"INFO:tensorflow:global_step/sec: 402.251\n",
"INFO:tensorflow:loss = 0.021794006, step = 3500 (0.250 sec)\n",
"INFO:tensorflow:global_step/sec: 395.529\n",
"INFO:tensorflow:loss = 0.037575096, step = 3600 (0.252 sec)\n",
"INFO:tensorflow:global_step/sec: 395.242\n",
"INFO:tensorflow:loss = 0.06161868, step = 3700 (0.251 sec)\n",
"INFO:tensorflow:global_step/sec: 402.537\n",
"INFO:tensorflow:loss = 0.02247855, step = 3800 (0.250 sec)\n",
"INFO:tensorflow:global_step/sec: 409.245\n",
"INFO:tensorflow:loss = 0.02687079, step = 3900 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 385.271\n",
"INFO:tensorflow:loss = 0.029623002, step = 4000 (0.258 sec)\n",
"INFO:tensorflow:global_step/sec: 393.757\n",
"INFO:tensorflow:loss = 0.020733567, step = 4100 (0.254 sec)\n",
"INFO:tensorflow:global_step/sec: 409.733\n",
"INFO:tensorflow:loss = 0.020996619, step = 4200 (0.246 sec)\n",
"INFO:tensorflow:global_step/sec: 411.107\n",
"INFO:tensorflow:loss = 0.059230465, step = 4300 (0.244 sec)\n",
"INFO:tensorflow:global_step/sec: 393.461\n",
"INFO:tensorflow:loss = 0.032371342, step = 4400 (0.253 sec)\n",
"INFO:tensorflow:global_step/sec: 387.777\n",
"INFO:tensorflow:loss = 0.06218221, step = 4500 (0.260 sec)\n",
"INFO:tensorflow:global_step/sec: 389.653\n",
"INFO:tensorflow:loss = 0.022168107, step = 4600 (0.255 sec)\n",
"INFO:tensorflow:global_step/sec: 381.999\n",
"INFO:tensorflow:loss = 0.02789202, step = 4700 (0.263 sec)\n",
"INFO:tensorflow:global_step/sec: 391.465\n",
"INFO:tensorflow:loss = 0.02288571, step = 4800 (0.255 sec)\n",
"INFO:tensorflow:global_step/sec: 396.413\n",
"INFO:tensorflow:loss = 0.034475923, step = 4900 (0.252 sec)\n",
"INFO:tensorflow:Calling checkpoint listeners before saving checkpoint 5000...\n",
"INFO:tensorflow:Saving checkpoints for 5000 into /tmp/tmpf039um29/model.ckpt.\n",
"INFO:tensorflow:Calling checkpoint listeners after saving checkpoint 5000...\n",
"INFO:tensorflow:Loss for final step: 0.02225061.\n"
],
"name": "stdout"
},
{
"output_type": "execute_result",
"data": {
"text/plain": [
"<tensorflow_estimator.python.estimator.canned.dnn.DNNClassifierV2 at 0x7fd2acef9978>"
]
},
"metadata": {
"tags": []
},
"execution_count": 135
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "LftIQMQQMCpb",
"colab_type": "text"
},
"source": [
"##### Evaluate the Classifier Model on test data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "VqkrdWhZIodg",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "7fa3354f-5a3f-445a-a9f8-648d75e162ba"
},
"source": [
"result = iris_dnn_classifier.evaluate(lambda: input_fn(iris_test, y_test, training=False))\n",
"clear_output()\n",
"print(\"Test Set Accuracy : %.2f \"%(result['accuracy']*100))"
],
"execution_count": 136,
"outputs": [
{
"output_type": "stream",
"text": [
"Test Set Accuracy : 96.67 \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "OnBfUEJiJxXY",
"colab_type": "text"
},
"source": [
"##### Evaluate the Classifier Model on the training data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "8a1JpCAhNdVR",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "91223f37-1d81-43e2-8d68-49493106482c"
},
"source": [
"result = iris_dnn_classifier.evaluate(lambda: input_fn(iris_train, y_train, training=False))\n",
"clear_output()\n",
"print(\"Train Set Accuracy : %.2f \"%(result['accuracy']*100))"
],
"execution_count": 137,
"outputs": [
{
"output_type": "stream",
"text": [
"Train Set Accuracy : 99.17 \n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "tTu_jWriN0Em",
"colab_type": "text"
},
"source": [
"##### Using the model for prediction"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "siH038gNOsBw",
"colab_type": "text"
},
"source": [
"Ask the user to input data on various features of the flower. The model will try to predict the species based on the data."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "IQMwyDa_jIxz",
"colab_type": "text"
},
"source": [
"Try with data like this:\n",
"\n",
"| Setosa | Versicolor | Virginica |\n",
"|---|---|---|\n",
"| 5.1 | 5.9 | 6.9 |\n",
"| 3.3 | 3.0 | 3.1 |\n",
"| 1.7 | 4.2 | 5.4 |\n",
"| 0.5 | 1.5 | 2.1 |\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "sJ4KyapzN55n",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 133
},
"outputId": "329155f0-74af-40ae-f276-890a780d6201"
},
"source": [
"predict = dict()\n",
"categories = ['SepalLength', 'SepalWidth', 'PetalLength', 'PetalWidth']\n",
"print('Please type info')\n",
"for category in categories:\n",
" valid = True\n",
" while valid:\n",
" val = input(category + ':')\n",
" if not val.isdigit():\n",
" valid = False\n",
" \n",
" predict[category] = [float(val)]\n",
"\n",
"def input_func(features, batch_size=256):\n",
" return tf.data.Dataset.from_tensor_slices(dict(features)).batch(batch_size)\n",
"\n",
"predictions = iris_dnn_classifier.predict(input_fn=lambda: input_func(predict))\n",
"clear_output()\n",
"for pred in predictions:\n",
" id = pred['class_ids'][0]\n",
" prob = pred['probabilities'][id]\n",
"\n",
" print('Prediction is: \"{}\" ({:.2f}%)'.format(species[id], prob * 100))"
],
"execution_count": 139,
"outputs": [
{
"output_type": "stream",
"text": [
"INFO:tensorflow:Calling model_fn.\n",
"INFO:tensorflow:Done calling model_fn.\n",
"INFO:tensorflow:Graph was finalized.\n",
"INFO:tensorflow:Restoring parameters from /tmp/tmpf039um29/model.ckpt-5000\n",
"INFO:tensorflow:Running local_init_op.\n",
"INFO:tensorflow:Done running local_init_op.\n",
"Prediction is: \"Versicolor\" (99.92%)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "bxP1xdM3WLF9",
"colab_type": "text"
},
"source": [
"So, we see that this model can much accurately predict the species of the flower based on user-provided feature data\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "7jtaXBHKWngG",
"colab_type": "text"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "kihZge-mdEd1",
"colab_type": "text"
},
"source": [
"## Basic Text Classification : Sentiment Analysis on an IMDB dataset"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qOu-aBfvuG-G",
"colab_type": "text"
},
"source": [
"### Import Packages"
]
},
{
"cell_type": "code",
"metadata": {
"id": "BYUGeBsKdDru",
"colab_type": "code",
"colab": {}
},
"source": [
"import matplotlib.pyplot as plt\n",
"import os\n",
"import re\n",
"import shutil\n",
"import string\n",
"import tensorflow as tf\n",
"\n",
"from tensorflow.keras import layers\n",
"from tensorflow.keras import losses\n",
"from tensorflow.keras import preprocessing\n",
"from tensorflow.keras.layers.experimental.preprocessing import TextVectorization"
],
"execution_count": 140,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_7X8VOGmuifv",
"colab_type": "text"
},
"source": [
"### Download and explore the IMDB dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "oRuz2AHRulRF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "32c81e72-25e1-4b6b-edd0-443c62dea0d2"
},
"source": [
"url = \"https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\"\n",
"\n",
"dataset = tf.keras.utils.get_file(\"aclImdb_v1.tar.gz\", url,\n",
" untar=True, cache_dir='.',\n",
" cache_subdir='')\n",
"\n",
"dataset_dir = os.path.join(os.path.dirname(dataset), 'aclImdb')\n",
"train_dir = os.path.join(dataset_dir, 'train')\n",
"\n",
"remove_dir = os.path.join(train_dir, 'unsup')\n",
"shutil.rmtree(remove_dir)"
],
"execution_count": 141,
"outputs": [
{
"output_type": "stream",
"text": [
"Downloading data from https://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz\n",
"84131840/84125825 [==============================] - 8s 0us/step\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ztdkS8g9yfIj",
"colab_type": "text"
},
"source": [
"#### Create the training dataset "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wgLeCwE5dDNg",
"colab_type": "text"
},
"source": [
"We will load the data off disk and prepare it into a format suitable for training. To do so, you will use the helpful [`text_dataset_from_directory`](https://www.tensorflow.org/api_docs/python/tf/keras/preprocessing/text_dataset_from_directory) utility, which expects a directory structure as follows.\n",
"\n",
"~~~\n",
"main_directory/\n",
"...class_a/\n",
"......a_text_1.txt\n",
"......a_text_2.txt\n",
"...class_b/\n",
"......b_text_1.txt\n",
"......b_text_2.txt\n",
"~~~\n",
"\n",
"To prepare a dataset for binary classification, we need two folders on disk, corresponding to `class_a` and `class_b`. These will be the positive and negative movie reviews, which can be found in `aclImdb/train/pos` and `aclImdb/train/neg`. As the IMDB dataset contains additional folders, we removed them before using this utility."
]
},
{
"cell_type": "code",
"metadata": {
"id": "H8pbC7JKdF-d",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "99ffec47-a92c-4811-fb8c-d86434df3c61"
},
"source": [
"batch = 42\n",
"seed = 42\n",
"\n",
"# training dataset\n",
"train_ds = preprocessing.text_dataset_from_directory('aclImdb/train',\n",
" batch_size = batch,\n",
" seed = seed,\n",
" validation_split = 0.2,\n",
" subset = 'training')\n"
],
"execution_count": 142,
"outputs": [
{
"output_type": "stream",
"text": [
"Found 25000 files belonging to 2 classes.\n",
"Using 20000 files for training.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Eg1A6-TpyNEQ",
"colab_type": "text"
},
"source": [
"#### Create a validation and test dataset"
]
},
{
"cell_type": "code",
"metadata": {
"id": "n2_OXaSGdGqm",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 50
},
"outputId": "b70e31a8-6c06-44ae-f6e8-bfc2b4d66dda"
},
"source": [
"# validation dataset\n",
"valid_ds = preprocessing.text_dataset_from_directory('aclImdb/train',\n",
" batch_size = batch,\n",
" seed = seed,\n",
" validation_split = 0.2,\n",
" subset = 'validation')"
],
"execution_count": 143,
"outputs": [
{
"output_type": "stream",
"text": [
"Found 25000 files belonging to 2 classes.\n",
"Using 5000 files for validation.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "N6DMhd4udHRn",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "96b5fc4d-9dae-4d40-924c-042c08ee7056"
},
"source": [
"# test dataset\n",
"test_ds = tf.keras.preprocessing.text_dataset_from_directory('aclImdb/test', \n",
" batch_size=batch)"
],
"execution_count": 145,
"outputs": [
{
"output_type": "stream",
"text": [
"Found 25000 files belonging to 2 classes.\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "yZ7XUwmzywvf",
"colab_type": "text"
},
"source": [
"#### Prepare the dataset for training\n",
"\n",
"Here, we will standardize, tokenize, and vectorize the data.\n",
"\n",
"- **Standardization** refers to preprocessing the text, typically to remove punctuation or HTML elements to simplify the dataset.\n",
"- **Tokenization** refers to splitting strings into tokens (for example, splitting a sentence into individual words, by splitting on whitespace).\n",
"- **Vectorization** refers to converting tokens into numbers so they can be fed into a neural network. All of these tasks can be accomplished with this layer."
]
},
{
"cell_type": "code",
"metadata": {
"id": "gTbWucn_dIM9",
"colab_type": "code",
"colab": {}
},
"source": [
"def custom_standardization(input_data):\n",
" # convert a string into lowercase\n",
" lowercase = tf.strings.lower(input_data)\n",
" # remove all the <br/> tags from the lowercase string\n",
" stripped_html = tf.strings.regex_replace(lowercase, '<br />', ' ')\n",
" # remove all punctuation from the stripped string\n",
" return tf.strings.regex_replace(stripped_html,\n",
" '[%s]' % re.escape(string.punctuation),'')\n",
"\n",
"max_features = 10000\n",
"sequence_length = 250\n",
"\n",
"# apply text vectorization\n",
"vectorize_layer = TextVectorization(\n",
" # use custom standardizer\n",
" standardize=custom_standardization,\n",
" max_tokens=max_features,\n",
" output_mode='int',\n",
" output_sequence_length=sequence_length)"
],
"execution_count": 146,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "8VW4dAdl9foo",
"colab_type": "text"
},
"source": [
"Now, we will call `adapt` to fit the state of the preprocessing layer to the dataset. This will cause the model to build an index of strings to integers."
]
},
{
"cell_type": "code",
"metadata": {
"id": "cBrZb8jodH7u",
"colab_type": "code",
"colab": {}
},
"source": [
"# Make a text-only dataset (without labels), then call adapt\n",
"train_text = train_ds.map(lambda x, y: x)\n",
"vectorize_layer.adapt(train_text)"
],
"execution_count": 147,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "WDAjf1kS-r0d",
"colab_type": "text"
},
"source": [
"create a function to see the result of using this layer to preprocess some data."
]
},
{
"cell_type": "code",
"metadata": {
"id": "hoSMeTyBdHBR",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 470
},
"outputId": "d984afe2-cf03-4d74-9e1b-d9311f17f35b"
},
"source": [
"def vectorize_text(text, label):\n",
" text = tf.expand_dims(text, -1)\n",
" return vectorize_layer(text), label\n",
"\n",
"# retrieve a batch (of 32 reviews and labels) from the dataset\n",
"text_batch, label_batch = next(iter(train_ds))\n",
"first_review, first_label = text_batch[0], label_batch[0]\n",
"print(\"Review\", first_review)\n",
"print(\"Label\", train_ds.class_names[first_label])\n",
"print(\"Vectorized review\", vectorize_text(first_review, first_label))"
],
"execution_count": 148,
"outputs": [
{
"output_type": "stream",
"text": [
"Review tf.Tensor(b'boring stuff we got here. His 5 minute shorts are better than this. know why? because there only 5 minutes and not 91 minutes or how ever long this is. <br /><br />The plot is kinda... eh.. the last half hour is alright the rest is boring and not funny =( I had my hopes up, the trailer made it look funny but the pace of this movie is pretty slow and sadly not funny. Just plain boring klaymen running into each other and trying to make us laugh.. not working.<br /><br />Maybe next time knox.<br /><br />Maybe re-cutting this movie and adding better scenes would do a lot of healing but for now its just not good.', shape=(), dtype=string)\n",
"Label neg\n",
"Vectorized review (<tf.Tensor: shape=(1, 250), dtype=int64, numpy=\n",
"array([[ 346, 524, 71, 184, 128, 24, 661, 909, 3043, 23, 122,\n",
" 70, 11, 118, 133, 84, 47, 61, 661, 226, 3, 21,\n",
" 1, 226, 41, 87, 121, 209, 11, 7, 2, 111, 7,\n",
" 1812, 7959, 2, 229, 363, 563, 7, 2757, 2, 351, 7,\n",
" 346, 3, 21, 160, 10, 66, 54, 1796, 56, 2, 1420,\n",
" 90, 9, 162, 160, 18, 2, 1052, 5, 11, 17, 7,\n",
" 179, 602, 3, 1030, 21, 160, 40, 1029, 346, 1, 633,\n",
" 81, 245, 78, 3, 258, 6, 96, 167, 472, 21, 773,\n",
" 278, 357, 58, 9521, 278, 1, 11, 17, 3, 2954, 122,\n",
" 136, 59, 82, 4, 171, 5, 8460, 18, 15, 148, 29,\n",
" 40, 21, 49, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,\n",
" 0, 0, 0, 0, 0, 0, 0, 0]])>, <tf.Tensor: shape=(), dtype=int32, numpy=0>)\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "YNPDsrXW_yeF",
"colab_type": "text"
},
"source": [
"each token has been replaced by an integer. We can lookup the token (string) that each integer corresponds to by calling `.get_vocabulary()` on the layer."
]
},
{
"cell_type": "code",
"metadata": {
"id": "AjVH5nn2dGeP",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "4fe1d31f-2c1f-4417-dec9-f99b3f4233be"
},
"source": [
"print(\"2166 ---> \",vectorize_layer.get_vocabulary()[2166])\n",
"print(\" 259 ---> \",vectorize_layer.get_vocabulary()[259 ])\n",
"print('Vocabulary size: {}'.format(len(vectorize_layer.get_vocabulary())))"
],
"execution_count": 149,
"outputs": [
{
"output_type": "stream",
"text": [
"2166 ---> grow\n",
" 259 ---> course\n",
"Vocabulary size: 10000\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ADQSgikkAIjd",
"colab_type": "text"
},
"source": [
"As a final preprocessing step, you will apply the `TextVectorization` layer you created earlier to the train, validation, and test dataset."
]
},
{
"cell_type": "code",
"metadata": {
"id": "IqBKXgNXdFY-",
"colab_type": "code",
"colab": {}
},
"source": [
"train_ds = train_ds.map(vectorize_text)\n",
"valid_ds = valid_ds.map(vectorize_text)\n",
"test_ds = test_ds.map(vectorize_text)"
],
"execution_count": 150,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "JFURiwzmATC3",
"colab_type": "text"
},
"source": [
"### Configure Dataset for performance\n",
"\n",
"These are two important methods we should use when loading data to make sure that I/O does not become blocking.\n",
"\n",
"`.cache()` keeps data in memory after it's loaded off disk. This will ensure the dataset does not become a bottleneck while training our model. If our dataset is too large to fit into memory, we can also use this method to create a performant on-disk cache, which is more efficient to read than many small files.\n",
"\n",
"`.prefetch()` overlaps data preprocessing and model execution while training.\n",
"\n",
"We can learn more about both methods, as well as how to cache data to disk in the [data performance guide](https://www.tensorflow.org/guide/data_performance)."
]
},
{
"cell_type": "code",
"metadata": {
"id": "q9-Mi5zBASUN",
"colab_type": "code",
"colab": {}
},
"source": [
"AUTOTUNE = -1\n",
"\n",
"train_ds = train_ds.cache().prefetch(buffer_size=AUTOTUNE)\n",
"valid_ds = valid_ds.cache().prefetch(buffer_size=AUTOTUNE)\n",
"test_ds = test_ds.cache().prefetch(buffer_size=AUTOTUNE)"
],
"execution_count": 151,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Q9HKXkj4AT5p",
"colab_type": "text"
},
"source": [
"### Build the Model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "krsAV6giHL4P",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 316
},
"outputId": "a80fcd83-09e0-4987-c4c0-2a3506815d75"
},
"source": [
"# initialize size of embedding dimensions to be 16\n",
"embedding_dim = 16\n",
"\n",
"model = tf.keras.Sequential([\n",
" # layers.Embedding : Turns positive integers (indexes) into dense vectors of fixed size.\n",
" layers.Embedding(max_features + 1, embedding_dim),\n",
" # layers.Dropout : apply Dropout regularization\n",
" layers.Dropout(0.2),\n",
" # layers.GlobalAveragePooling2D : Global average pooling operation for spatial data\n",
" layers.GlobalAveragePooling1D(),\n",
" layers.Dropout(0.2),\n",
" layers.Dense(1)])\n",
"\n",
"model.summary()"
],
"execution_count": 152,
"outputs": [
{
"output_type": "stream",
"text": [
"Model: \"sequential_2\"\n",
"_________________________________________________________________\n",
"Layer (type) Output Shape Param # \n",
"=================================================================\n",
"embedding (Embedding) (None, None, 16) 160016 \n",
"_________________________________________________________________\n",
"dropout (Dropout) (None, None, 16) 0 \n",
"_________________________________________________________________\n",
"global_average_pooling1d (Gl (None, 16) 0 \n",
"_________________________________________________________________\n",
"dropout_1 (Dropout) (None, 16) 0 \n",
"_________________________________________________________________\n",
"dense_3 (Dense) (None, 1) 17 \n",
"=================================================================\n",
"Total params: 160,033\n",
"Trainable params: 160,033\n",
"Non-trainable params: 0\n",
"_________________________________________________________________\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "G554OPzMPaNN",
"colab_type": "text"
},
"source": [
"#### Loss function and optimizer\n",
"\n",
"A model needs a loss function and an optimizer for training. Since this is a binary classification problem and the model outputs a probability (a single-unit layer with a sigmoid activation), we'll use `losses.BinaryCrossentropy` loss function."
]
},
{
"cell_type": "code",
"metadata": {
"id": "j0T4PbQ1PQHS",
"colab_type": "code",
"colab": {}
},
"source": [
"model.compile(loss=losses.BinaryCrossentropy(from_logits=True),\n",
" optimizer='adam',\n",
" metrics=tf.metrics.BinaryAccuracy(threshold=0.0))"
],
"execution_count": 153,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "ytzxNz4vPfEd",
"colab_type": "text"
},
"source": [
"### Training the model on the training data\n",
"\n",
"train the model by passing the `dataset` object to the `fit` method."
]
},
{
"cell_type": "code",
"metadata": {
"id": "83Cz151GPyoK",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 370
},
"outputId": "4b4c7894-6050-4862-eee0-86208eb9e2a3"
},
"source": [
"train_hist = model.fit(train_ds,\n",
" validation_data = valid_ds,\n",
" epochs = 10)"
],
"execution_count": 155,
"outputs": [
{
"output_type": "stream",
"text": [
"Epoch 1/10\n",
"477/477 [==============================] - 12s 26ms/step - loss: 0.6733 - binary_accuracy: 0.6831 - val_loss: 0.6378 - val_binary_accuracy: 0.7624\n",
"Epoch 2/10\n",
"477/477 [==============================] - 3s 5ms/step - loss: 0.5825 - binary_accuracy: 0.7861 - val_loss: 0.5370 - val_binary_accuracy: 0.8088\n",
"Epoch 3/10\n",
"477/477 [==============================] - 3s 6ms/step - loss: 0.4862 - binary_accuracy: 0.8293 - val_loss: 0.4574 - val_binary_accuracy: 0.8356\n",
"Epoch 4/10\n",
"477/477 [==============================] - 3s 6ms/step - loss: 0.4148 - binary_accuracy: 0.8561 - val_loss: 0.4046 - val_binary_accuracy: 0.8522\n",
"Epoch 5/10\n",
"477/477 [==============================] - 3s 5ms/step - loss: 0.3677 - binary_accuracy: 0.8692 - val_loss: 0.3699 - val_binary_accuracy: 0.8620\n",
"Epoch 6/10\n",
"477/477 [==============================] - 3s 5ms/step - loss: 0.3333 - binary_accuracy: 0.8794 - val_loss: 0.3464 - val_binary_accuracy: 0.8670\n",
"Epoch 7/10\n",
"477/477 [==============================] - 3s 6ms/step - loss: 0.3082 - binary_accuracy: 0.8885 - val_loss: 0.3296 - val_binary_accuracy: 0.8714\n",
"Epoch 8/10\n",
"477/477 [==============================] - 3s 5ms/step - loss: 0.2875 - binary_accuracy: 0.8958 - val_loss: 0.3173 - val_binary_accuracy: 0.8720\n",
"Epoch 9/10\n",
"477/477 [==============================] - 3s 5ms/step - loss: 0.2712 - binary_accuracy: 0.9020 - val_loss: 0.3081 - val_binary_accuracy: 0.8728\n",
"Epoch 10/10\n",
"477/477 [==============================] - 3s 5ms/step - loss: 0.2561 - binary_accuracy: 0.9081 - val_loss: 0.3009 - val_binary_accuracy: 0.8766\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "vDV8_ULIRE8C",
"colab_type": "text"
},
"source": [
"### Evaluate the model on the training data"
]
},
{
"cell_type": "code",
"metadata": {
"id": "7I913ELpQ8lC",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 66
},
"outputId": "0be83132-8c3b-4157-e98d-c3fae1b0e4db"
},
"source": [
"loss, accuracy = model.evaluate(test_ds)\n",
"\n",
"print(\"Loss: \", loss)\n",
"print(\"Accuracy: \", accuracy)"
],
"execution_count": 156,
"outputs": [
{
"output_type": "stream",
"text": [
"596/596 [==============================] - 8s 14ms/step - loss: 0.3185 - binary_accuracy: 0.8706\n",
"Loss: 0.3185163140296936\n",
"Accuracy: 0.8705999851226807\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_1DAa8IFRQsQ",
"colab_type": "text"
},
"source": [
"### Accuracy vs. Loss over time\n",
"\n",
"`model.fit()` returns a `History` object that contains a dictionary with everything that happened during training:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "e3Nwgt7RQ9Hc",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "2cf7bd80-444d-4fd5-dbfb-3a743d757185"
},
"source": [
"train_hist_dict = train_hist.history\n",
"train_hist_dict.keys()"
],
"execution_count": 157,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"dict_keys(['loss', 'binary_accuracy', 'val_loss', 'val_binary_accuracy'])"
]
},
"metadata": {
"tags": []
},
"execution_count": 157
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "HNW5_tESR8Zm",
"colab_type": "text"
},
"source": [
"There are four entries: one for each monitored metric during training and validation. We can use these to plot the training and validation loss for comparison, as well as the training and validation accuracy:"
]
},
{
"cell_type": "code",
"metadata": {
"id": "0YGiLE7zQ83f",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 294
},
"outputId": "7be7081b-5d65-428b-cff6-aa50f365deee"
},
"source": [
"acc = train_hist_dict['binary_accuracy']\n",
"val_acc = train_hist_dict['val_binary_accuracy']\n",
"loss = train_hist_dict['loss']\n",
"val_loss = train_hist_dict['val_loss']\n",
"\n",
"epochs = range(1, len(acc) + 1)\n",
"\n",
"# \"bo\" is for \"blue dot\"\n",
"plt.plot(epochs, loss, 'bo', label='Training loss')\n",
"# b is for \"solid blue line\"\n",
"plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
"plt.title('Training and validation loss')\n",
"plt.xlabel('Epochs')\n",
"plt.ylabel('Loss')\n",
"plt.legend()\n",
"\n",
"plt.show()"
],
"execution_count": 158,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3xU1bn/8c8T7gii3FSugRakXMNdQRC1PYpQUPGGqUo5ilC8oVVRVDi2WFs5/iwVrahFbWPRQ1vEeq0KglorSBFEsVIINl4R5WZAuTy/P9aETEISEsjMnmS+79drXpnZs2fPk0Hnm73WXmuZuyMiIukrI+oCREQkWgoCEZE0pyAQEUlzCgIRkTSnIBARSXMKAhGRNKcgkEplZs+a2cWVvW+UzCzXzL6fgOO6mX03dv+3ZnZLefY9iPfJNrMXDrbOMo47xMzyKvu4knw1oy5Aomdm2+Me1ge+AfbEHl/m7jnlPZa7D03EvtWdu4+vjOOYWSawHqjl7rtjx84Byv1vKOlHQSC4e4OC+2aWC1zi7i8W38/MahZ8uYhI9aGmISlVwam/md1gZp8Cc8zsSDP7q5ltNLOvYvdbxb1mkZldErs/xsxeNbMZsX3Xm9nQg9y3nZktNrNtZvaimc0ysz+UUnd5avyZmb0WO94LZtY07vkLzWyDmW0ysyllfD79zexTM6sRt+1MM1sZu9/PzP5uZpvN7BMzu8fMapdyrIfN7Odxj6+LveZjMxtbbN9hZvZPM9tqZv8xs2lxTy+O/dxsZtvN7PiCzzbu9QPMbKmZbYn9HFDez6YsZva92Os3m9lqMxsR99zpZvZu7JgfmdlPY9ubxv59NpvZl2a2xMz0vZRk+sDlQI4GGgNtgXGE/2bmxB63AXYA95Tx+v7A+0BT4FfAQ2ZmB7HvY8CbQBNgGnBhGe9ZnhovAH4MNAdqAwVfTJ2B+2LHbxF7v1aUwN3/AXwNnFzsuI/F7u8BJsV+n+OBU4CflFE3sRpOi9XzA6ADULx/4mvgIuAIYBgwwczOiD03OPbzCHdv4O5/L3bsxsDTwMzY73YX8LSZNSn2O+z32Ryg5lrAU8ALsdddAeSY2bGxXR4iNDM2BLoCL8e2XwvkAc2Ao4CbAM17k2QKAjmQvcBUd//G3Xe4+yZ3/5O757v7NmA6cGIZr9/g7g+4+x7gEeAYwv/w5d7XzNoAfYFb3f1bd38VWFDaG5azxjnu/i933wE8AWTFtp8N/NXdF7v7N8Atsc+gNH8ERgOYWUPg9Ng23P0td3/D3Xe7ey5wfwl1lOTcWH3vuPvXhOCL//0Wufsqd9/r7itj71ee40IIjg/c/fexuv4IrAF+GLdPaZ9NWY4DGgB3xP6NXgb+SuyzAXYBnc3scHf/yt2Xx20/Bmjr7rvcfYlrArSkUxDIgWx0950FD8ysvpndH2s62UpoijgivnmkmE8L7rh7fuxugwru2wL4Mm4bwH9KK7icNX4adz8/rqYW8ceOfRFvKu29CH/9n2VmdYCzgOXuviFWR8dYs8ensTpuJ5wdHEiRGoANxX6//ma2MNb0tQUYX87jFhx7Q7FtG4CWcY9L+2wOWLO7x4dm/HFHEUJyg5m9YmbHx7bfCawFXjCzdWY2uXy/hlQmBYEcSPG/zq4FjgX6u/vhFDZFlNbcUxk+ARqbWf24ba3L2P9Qavwk/tix92xS2s7u/i7hC28oRZuFIDQxrQE6xOq46WBqIDRvxXuMcEbU2t0bAb+NO+6B/pr+mNBkFq8N8FE56jrQcVsXa9/fd1x3X+ruIwnNRvMJZxq4+zZ3v9bd2wMjgGvM7JRDrEUqSEEgFdWQ0Oa+OdbePDXRbxj7C3sZMM3Masf+mvxhGS85lBrnAcPN7IRYx+5tHPj/k8eAqwiB83/F6tgKbDezTsCEctbwBDDGzDrHgqh4/Q0JZ0g7zawfIYAKbCQ0ZbUv5djPAB3N7AIzq2lm5wGdCc04h+IfhLOH682slpkNIfwbzY39m2WbWSN330X4TPYCmNlwM/turC9oC6FfpaymOEkABYFU1N1APeAL4A3guSS9bzahw3UT8HPgccJ4h5IcdI3uvhqYSPhy/wT4itCZWZaCNvqX3f2LuO0/JXxJbwMeiNVcnhqejf0OLxOaTV4utstPgNvMbBtwK7G/rmOvzSf0ibwWuxLnuGLH3gQMJ5w1bQKuB4YXq7vC3P1bwhf/UMLnfi9wkbuvie1yIZAbayIbT/j3hNAZ/iKwHfg7cK+7LzyUWqTiTP0yUhWZ2ePAGndP+BmJSHWnMwKpEsysr5l9x8wyYpdXjiS0NYvIIdLIYqkqjgb+TOi4zQMmuPs/oy1JpHpQ05CISJpT05CISJqrck1DTZs29czMzKjLEBGpUt56660v3L1ZSc9VuSDIzMxk2bJlUZchIlKlmFnxEeX7qGlIRCTNKQhERNKcgkBEJM1VuT4CEUm+Xbt2kZeXx86dOw+8s0Sqbt26tGrVilq1apX7NQoCETmgvLw8GjZsSGZmJqWvKyRRc3c2bdpEXl4e7dq1K/fr0qJpKCcHMjMhIyP8zNEy3iIVsnPnTpo0aaIQSHFmRpMmTSp85lbtzwhycmDcOMiPLWmyYUN4DJCdXfrrRKQohUDVcDD/TtX+jGDKlMIQKJCfH7aLiEgaBMGHH1Zsu4iknk2bNpGVlUVWVhZHH300LVu23Pf422+/LfO1y5Yt48orrzzgewwYMKBSal20aBHDhw+vlGMlS7UPgjbFF/k7wHYROXSV3S/XpEkTVqxYwYoVKxg/fjyTJk3a97h27drs3r271Nf26dOHmTNnHvA9Xn/99UMrsgqr9kEwfTrUr190W/36YbuIVL6CfrkNG8C9sF+usi/SGDNmDOPHj6d///5cf/31vPnmmxx//PH07NmTAQMG8P777wNF/0KfNm0aY8eOZciQIbRv375IQDRo0GDf/kOGDOHss8+mU6dOZGdnUzBL8zPPPEOnTp3o3bs3V1555QH/8v/yyy8544wz6N69O8cddxwrV64E4JVXXtl3RtOzZ0+2bdvGJ598wuDBg8nKyqJr164sWbKkcj+wMlT7zuKCDuEpU0JzUJs2IQTUUSySGGX1y1X2/3d5eXm8/vrr1KhRg61bt7JkyRJq1qzJiy++yE033cSf/vSn/V6zZs0aFi5cyLZt2zj22GOZMGHCftfc//Of/2T16tW0aNGCgQMH8tprr9GnTx8uu+wyFi9eTLt27Rg9evQB65s6dSo9e/Zk/vz5vPzyy1x00UWsWLGCGTNmMGvWLAYOHMj27dupW7cus2fP5tRTT2XKlCns2bOH/OIfYgJV+yCA8B+fvvhFkiOZ/XLnnHMONWrUAGDLli1cfPHFfPDBB5gZu3btKvE1w4YNo06dOtSpU4fmzZvz2Wef0apVqyL79OvXb9+2rKwscnNzadCgAe3bt993ff7o0aOZPXt2mfW9+uqr+8Lo5JNPZtOmTWzdupWBAwdyzTXXkJ2dzVlnnUWrVq3o27cvY8eOZdeuXZxxxhlkZWUd0mdTEdW+aUhEkiuZ/XKHHXbYvvu33HILJ510Eu+88w5PPfVUqdfS16lTZ9/9GjVqlNi/UJ59DsXkyZN58MEH2bFjBwMHDmTNmjUMHjyYxYsX07JlS8aMGcOjjz5aqe9ZFgWBiFSqqPrltmzZQsuWLQF4+OGHK/34xx57LOvWrSM3NxeAxx9//ICvGTRoEDmxzpFFixbRtGlTDj/8cP7973/TrVs3brjhBvr27cuaNWvYsGEDRx11FJdeeimXXHIJy5cvr/TfoTQKAhGpVNnZMHs2tG0LZuHn7NmJb569/vrrufHGG+nZs2el/wUPUK9ePe69915OO+00evfuTcOGDWnUqFGZr5k2bRpvvfUW3bt3Z/LkyTzyyCMA3H333XTt2pXu3btTq1Ythg4dyqJFi+jRowc9e/bk8ccf56qrrqr036E0VW7N4j59+rgWphFJrvfee4/vfe97UZcRue3bt9OgQQPcnYkTJ9KhQwcmTZoUdVn7Kenfy8zecvc+Je2vMwIRkXJ64IEHyMrKokuXLmzZsoXLLrss6pIqRVpcNSQiUhkmTZqUkmcAh0pnBCIiaU5BICKS5hQEIiJpTkEgIpLmFAQikvJOOukknn/++SLb7r77biZMmFDqa4YMGULBpeann346mzdv3m+fadOmMWPGjDLfe/78+bz77rv7Ht966628+OKLFSm/RKk0XXXaBMGzz8KwYVDK9CMiksJGjx7N3Llzi2ybO3duuSZ+gzBr6BFHHHFQ7108CG677Ta+//3vH9SxUlXaBEF+PjzzDEybFnUlIlJRZ599Nk8//fS+RWhyc3P5+OOPGTRoEBMmTKBPnz506dKFqVOnlvj6zMxMvvjiCwCmT59Ox44dOeGEE/ZNVQ1hjEDfvn3p0aMHo0aNIj8/n9dff50FCxZw3XXXkZWVxb///W/GjBnDvHnzAHjppZfo2bMn3bp1Y+zYsXzzzTf73m/q1Kn06tWLbt26sWbNmjJ/v6inq06bcQSjRsHYsfCLX8Cpp8LgwVFXJFI1XX01rFhRucfMyoK77y79+caNG9OvXz+effZZRo4cydy5czn33HMxM6ZPn07jxo3Zs2cPp5xyCitXrqR79+4lHuett95i7ty5rFixgt27d9OrVy969+4NwFlnncWll14KwM0338xDDz3EFVdcwYgRIxg+fDhnn312kWPt3LmTMWPG8NJLL9GxY0cuuugi7rvvPq6++moAmjZtyvLly7n33nuZMWMGDz74YKm/X9TTVafNGQHAr38N7dvDhRdCCc2FIpLC4puH4puFnnjiCXr16kXPnj1ZvXp1kWac4pYsWcKZZ55J/fr1OfzwwxkxYsS+59555x0GDRpEt27dyMnJYfXq1WXW8/7779OuXTs6duwIwMUXX8zixYv3PX/WWWcB0Lt3730T1ZXm1Vdf5cILLwRKnq565syZbN68mZo1a9K3b1/mzJnDtGnTWLVqFQ0bNizz2OWRNmcEAA0ahFWSBg6EiRMrf8UkkXRQ1l/uiTRy5EgmTZrE8uXLyc/Pp3fv3qxfv54ZM2awdOlSjjzySMaMGVPq9NMHMmbMGObPn0+PHj14+OGHWbRo0SHVWzCV9aFMYz158mSGDRvGM888w8CBA3n++ef3TVf99NNPM2bMGK655houuuiiQ6o1rc4IAPr3h6lT4bHHFAQiVUmDBg046aSTGDt27L6zga1bt3LYYYfRqFEjPvvsM5599tkyjzF48GDmz5/Pjh072LZtG0899dS+57Zt28YxxxzDrl279k0dDdCwYUO2bdu237GOPfZYcnNzWbt2LQC///3vOfHEEw/qd4t6uuq0OiMocOON8Nxz8JOfhLODzMyoKxKR8hg9ejRnnnnmviaigmmbO3XqROvWrRk4cGCZr+/VqxfnnXcePXr0oHnz5vTt23ffcz/72c/o378/zZo1o3///vu+/M8//3wuvfRSZs6cua+TGKBu3brMmTOHc845h927d9O3b1/Gjx9/UL9XwVrK3bt3p379+kWmq164cCEZGRl06dKFoUOHMnfuXO68805q1apFgwYNKmUBm7Sdhnr9eujRI3RSLVwIsdXuRKQEmoa6atE01OXUrh3MmgVLlsAvfxl1NSIi0UnbIAD40Y/gvPNCn8HSpVFXIyISjbQOAjO47z445piwjN727VFXJJK6qlozcro6mH+ntA4CgCOPhEcfhbVr4Zproq5GJDXVrVuXTZs2KQxSnLuzadMm6tatW6HXpeVVQ8UNGQLXXx/6CoYOhTPPjLoikdTSqlUr8vLy2LhxY9SlyAHUrVuXVq1aVeg1aXvVUHHffgvHHw8bNsDKldCiRaW/hYhIZHTVUDnUrh0GmOXnw49/DHv3Rl2RiEhyKAjidOoEd90FL7wAv/lN1NWIiCRHQoPAzE4zs/fNbK2ZTS5ln3PN7F0zW21mjyWynvK47DL44Q/hhhtg1aqoqxERSbyEBYGZ1QBmAUOBzsBoM+tcbJ8OwI3AQHfvAlydqHrKywwefBAaNYILLoCDnL9KRKTKSOQZQT9grbuvc/dvgbnAyGL7XArMcvevANz98wTWU27Nm8PDD8M774R5iUREqrNEBkFL4D9xj/Ni2+J1BDqa2Wtm9oaZnVbSgcxsnJktM7Nlybp8behQuPzyMOXuCy8k5S1FRCIRdWdxTaADMAQYDTxgZvstLOrus929j7v3adasWdKK+9WvoHNnuPhiiK1yJyJS7SQyCD4CWsc9bhXbFi8PWODuu9x9PfAvQjCkhHr1wiWlX34Jl1wCVWzIhYhIuSQyCJYCHcysnZnVBs4HFhTbZz7hbAAza0poKlqXwJoqLCsLbr8dnnwydCKLiFQ3CQsCd98NXA48D7wHPOHuq83sNjMrWCj0eWCTmb0LLASuc/dNiarpYE2aBKecEhbt/te/oq5GRKRyJbSPwN2fcfeO7v4dd58e23aruy+I3Xd3v8bdO7t7N3efm8h6DlZGBjzyCNSpE2Yp3bXr4I6TkxNWQ8vICD+1VKaIpIKoO4urjJYt4YEHYNkymDat4q/PyYFx48JcRu7h57hxCgMRiZ6CoAJGjQrzEP3iF2Fls4qYMiXMYxQvPz9sFxGJkoKggn79a2jfHi68ELZsKf/rPvywYttFRJJFQVBBDRuG5py8PJg4sfyva9OmYttFRJJFQXAQ+veHW28NgfBYOafJmz4d6tcvuq1+/bBdRCRKCoKDdNNNMGAATJgQOn4PJDsbZs+Gtm3DxHZt24bH2dmJr1VEpCxaoewQrFsXBpxlZcHChVCjRtQViYiUTCuUJUj79nDPPeEKol/+MupqREQOjoLgEF14IZx7LkydGsYYiIhUNQqCQ2QGv/0tHH10WMjm66+jrkhEpGIUBJXgyCPh97+HtWvDvEQiIlWJgqCSDBkC110XpqGYPz/qakREyk9BUIl+9jPo1SusXfDJJ1FXIyJSPgqCSlS7dhhklp8PY8bA3r1RVyQicmAKgkrWqRP87/+GdY5/85uoqxEROTAFQQKMHw/Dh8MNN8CqVVFXIyJSNgVBApjBQw9Bo0ZhComdO6OuSESkdAqCBGneHObMCWcEN94YdTUiIqVTECTQ6afD5ZfD3XeHPgMRkVSkIEiwX/0KOncOVxF98UXU1YiI7E9BkGD16oVLSr/4Ai69NKxXLCKSShQESZCVFdY5nj8/dCKLiKQSBUGSTJoEp5wCV10FH3wQdTUiIoUUBEmSkQEPPwx16sB558H27VFXJCISKAiSqFWrMEvp22/DOefArl1RVyQioiBIumHD4P774bnnwuR06jwWkajVjLqAdFQwO+mtt0KLFqEjWUQkKgqCiNx8M3z0EdxxRwiDK66IuiIRSVcKgoiYwaxZ8Nln4Uqio48O/QYiIsmmPoII1agBjz0GAwbAj34EixZFXZGIpCMFQcTq1YMFC+A734GRI2HlyqgrEpF0oyBIAY0bh6uIGjaEoUPhww+jrkhE0omCIEW0aRPC4Ouv4dRTYdOmqCsSkXShIEghXbvCk0/C+vXwwx+GtY9FRBJNQZBiTjwxzFb6xhswejTs3h11RSJS3SkIUtCoUWHh+wULYOJEjT4WkcTSOIIUNXEifPwx3H57GHA2dWrUFYlIdaUgSGE//3kIg2nTQhhcemnUFYlIdZTQpiEzO83M3jeztWY2uYTnx5jZRjNbEbtdksh6qhozmD07XFI6fnxoKhIRqWwJCwIzqwHMAoYCnYHRZta5hF0fd/es2O3BRNVTVdWqBf/3f9C7d1jH4PXXo65IRKqbRJ4R9APWuvs6d/8WmAuMTOD7VVuHHQZPPw2tW4fLSt97L+qKRKQ6SWQQtAT+E/c4L7atuFFmttLM5plZ65IOZGbjzGyZmS3buHFjImpNec2ahQFntWrBaaeFmUtFRCpD1JePPgVkunt34G/AIyXt5O6z3b2Pu/dp1qxZUgtMJe3bw7PPwpdfhn6DzZujrkhEqoNEBsFHQPxf+K1i2/Zx903u/k3s4YNA7wTWUy307Al//jOsWQNnnAE7d0ZdkYhUdYkMgqVABzNrZ2a1gfOBIte9mNkxcQ9HAGr9Locf/AAefhheeQUuugj27o26IhGpyhIWBO6+G7gceJ7wBf+Eu682s9vMbERstyvNbLWZvQ1cCYxJVD3VzQUXwIwZ4Yqiq6+u2OjjnBzIzISMjPAzJydRVYpIVWBexeYv6NOnjy9btizqMlLGtdfCXXeFJS9vuOHA++fkwLhxRSe0q18/jFfIzk5cnSISLTN7y937lPRc1J3FcojuvDNMTjd5Mjz66IH3nzJl/1lN8/PDdhFJT5pioorLyIA5c+Dzz+G//xuaNw+Xl5amtEVvtBiOSPrSGUE1UKdOuJKoa1c4+2xYurT0fdu0qdh2Ean+yhUEZnaYmWXE7nc0sxFmViuxpUlFHH54GGPQrBkMGwZr15a83/TpoU8gXv36YbuIpKfynhEsBuqaWUvgBeBC4OFEFSUH5+ij4fnnwxVEp54Kn322/z7Z2aFjuG3bMKld27bqKBZJd+UNAnP3fOAs4F53Pwfokriy5GB17Ah//St8+imcfjps27b/PtnZkJsbxh/k5ioERNJduYPAzI4HsoGnY9tqJKYkOVT9+4fxBW+/HfoMvv026opEJJWVNwiuBm4E/hIbFNYeWJi4suRQnX46PPAAvPACjB2r0cciUrpyXT7q7q8ArwDEOo2/cPcrE1mYHLof/ziscHbzzWGFs1/9KuqKRCQVlfeqocfM7HAzOwx4B3jXzK5LbGlSGW66CX7ykzDw7O67o65GRFJReZuGOrv7VuAM4FmgHeHKIUlxZjBzJpx1FkyaBHPnRl2RiKSa8gZBrdi4gTOABe6+C6hakxSlsRo1whxDgwaF2UpffjnqikQklZQ3CO4HcoHDgMVm1hbYmqiipPLVrQtPPhkuLz3jDFixIuqKRCRVlCsI3H2mu7d099M92ACclODapJIdeWRY7rJRo7DCWW5u1BWJSCoob2dxIzO7q2DdYDP7X8LZgVQxrVqFMNi5E447Dl58MeqKRCRq5W0a+h2wDTg3dtsKzElUUZJYXbrAkiXQuDH813/BLbfA7t1RVyUiUSlvEHzH3ae6+7rY7X+A9oksTBKra9cwS+mPfww//zmcfDLk5UVdlYhEobxBsMPMTih4YGYDgR2JKUmS5bDD4KGH4A9/gH/+E7Ky4OmnD/w6EaleyhsE44FZZpZrZrnAPcBlCatKkio7G956C1q3huHDw/KXmp9IJH2U96qht929B9Ad6O7uPYGTE1qZJFXHjvD3v8PEiWEN5BNOgHXroq5KRJKhQiuUufvW2AhjgGsSUI9EqG5duOce+NOf4F//gp49Yd68qKsSkUQ7lKUqrdKqkJRy1lmhz+B734NzzglzFe3cGXVVIpIohxIEmmKiGmvXLlxiet11cN99YY2DNWuirkpEEqHMIDCzbWa2tYTbNqBFkmqUiNSqFaaufuaZMJ11nz7w+99HXZWIVLYyg8DdG7r74SXcGrp7udYykKpv6NAwN1Hv3mHSujFjYPv2qKsSkcpyKE1DkkZatoSXXoJbb4VHH4W+fWHlyqirEpHKoCCQcqtZE/7nf8L8RJs3h36D++8HV2+RSJWmIJAKO/lkePttGDwYxo+H88+HLVuirkpEDpaCQA5K8+bw7LNwxx1h3EGvXmHuIhGpehQEctAyMuCGG2Dx4jB76cCB8P/+n5qKRKoaBYEcsgEDwgC000+Ha66BkSNh06aoqxKR8lIQSKVo3Bj+8heYOROefz7MZPrqq1FXJSLloSCQSmMGV1wBr78OderAkCFw++2wd2/J++fkQGZmaGLKzAyPRST5FARS6Xr3huXLwzxFU6bAaafBZ58V3ScnB8aNgw0bQp/Chg3hscJAJPkUBJIQhx8Ojz0GDzwQ5izq0aPo+shTpkB+ftHX5OeH7SKSXAoCSRgzuOSScFlpkyZhfeSbbw5XGH34YcmvKW27iCSOgkASrmtXePPNsD7y9Olw0knQopQpC9u0SW5tIqIgkCSJXx95xYowErlOnaL71K8fgkJEkktBIElVsD7yd78L33wDDRuG7W3bwuzZ4XkRSa6EBoGZnWZm75vZWjObXMZ+o8zMzaxPIuuR1FCwPvLll8O2bdC6dVgAZ9SoqCsTSU8JCwIzqwHMAoYCnYHRZta5hP0aAlcB/0hULZJ66taF3/wG/va30C9w+eVhVbS77oKvv466OpH0ksgzgn7AWndf5+7fAnOBkSXs9zPgl4BWxU1D3/9+uLx04ULo3BmuvTYMLrv9dti6NerqRNJDIoOgJfCfuMd5sW37mFkvoLW7P13WgcxsnJktM7NlGzdurPxKJVJmYRTySy/Ba69Bv35hPEHbtjB1Knz5ZdQVilRvkXUWm1kGcBdw7YH2dffZ7t7H3fs0a9Ys8cVJZAYMgKefhmXLwmWmt90WAmHyZPj886irE6meEhkEHwGt4x63im0r0BDoCiwys1zgOGCBOowFwjQVf/4zrFoFP/wh3HlnaDK6+mr46KMDvlxEKiCRQbAU6GBm7cysNnA+sKDgSXff4u5N3T3T3TOBN4AR7r4sgTVJFdO1a5iq4r334Lzz4J57oH17mDABcnOjrk6kekhYELj7buBy4HngPeAJd19tZreZ2YhEva9UTx07wpw58MEHYYTy734HHTqE+x98EHV1IlWbeRVbTqpPnz6+bJlOGtJdXh7MmAH33w/ffhvOFqZMgS5doq5MJDWZ2VvuXmLTu0YWS5XUqhXcfXdoHvrpT2HBgtCMNGpUmAJbRMpPQSBV2lFHwS9/GdYzuOWWcAlq794wfDi88UbU1YlUDQoCqRaaNAmXmm7YECaue+MNOP74MGDtlVfC4jciUjIFgVQrjRrBTTeFJqMZM+Cdd8JgtcGDw1rKCgSR/SkIpFpq0CBMV7F+fZjTKDc3LJnZv3/oT1AgiBRSEEi1Vq9emNDu3/8Oy2Zu2gQjR0JWFjzxBOzZE3WFItFTEEhaqF07LJv5/vvw6KNhLYTzzgtXGk2YEKaxyMgIo5dzcqKuViS5FASSVmrWhAsvhNWr4fHHw03kHrkAAAxySURBVJTXv/1tWCvZPXQ2jxunMJD0oiCQtFSjBpx7bpj5tLj8fJg4McxzpL4ESQcKAklr//lPydu3bIHu3eF734Obbw7rLCsUpLpSEEhaa9Om5O2tWsF994Wfv/gF9OwZ5ju68cYwclmhINWJgkDS2vTpUL9+0W3168Mdd8D48fDii/DppzB7dlhK8847w8jl734XbrgBli5VKEjVpyCQtJadHb7k27YN/QVt24bH2dmF+zRrBpdeCi+8AJ99Bg8+GM4O7rorrKbWrl2Y7+iNNxQKUjVp9lGRg/TVV/DkkzBvXgiJXbugdesw8d3ZZ4cpLjL0p5akiLJmH1UQiFSCzZvhqadCKDz3XJgau0WLEArnnBOW4KxRI+oqJZ1pGmqRBDviiDA+4cknYePGMA6hX7/QzDR4cOh0njgRFi3SaGZJPQoCkUp2+OFwwQXwl7+EUJg7FwYODCusnXRSOFOYMCFMmb17d9TViigIRBKqYcMwlcW8eSEUnngizIb66KNhiuxjjgkjmQv6GESioD4CkQjk54e+hHnzQt/C9u3QuDGMGBGCYtCgcDVSSSOfRQ6GOotFUtiOHeGMYN48+OtfQ8czhLOFE04IoXDCCWGkszqc5WApCESqiL174d13YckSePXVcPvww/Bcw4bhktSCYOjXb//BcCKlURCIVGEfflgYCq++GlZdc4datcIo5xNOCLeBA6Fp06irlVSlIBBJcTk5MGVK+NJv0yZMfRE/ujneV1/B668XBsObb4ZxCxAmySsIhhNOUD+DFFIQiKSwnJxw5VB+fuG2+vX3n+qiNDt3wrJlhcHw2muF/QwtWhSGwqBB0K2b+hnSlYJAJIVlZoYFcYpr2zastVxRe/eGhXcKgmHJksLpths2DKOcC4KhX7+wnKdUfwoCkRSWkVHyZHVm4Uu9MpSnn2HQoNDP0KRJ5bynpBYFgUgKq+wzgvKI72dYsiRMpx3fz3D88aEZqWvX8LN5c/U1VHUKApEUdqh9BJWheD/Dm2+GkdAFmjYNoVBw69YNunSBRo2SU58curKCoGayixGRogq+7Mt71VAi1K1b2Klc4PPPQxNS/O2RR2DbtsJ9WrcuGg5du0KnTup3qGp0RiAi5eYewio+HFatgvfeK2xaysgIK7jFh0PXrmFbTf3pGRmdEYhIpShYxa1tWxg2rHD77t2wdu3+ATF/fmGHd+3aof8hPhy6dg1nQOp/iJbOCEQkYXbsgDVriobDO+8UXs4K4ZLW+GAouDVvHl3d1ZE6i0UkpWzZEsY6xAfEqlWwaVPhPs2bw7HHhtHR8bf27cNAOS0DWjFqGhKRlNKoURjYNmBA4Tb3oh3Uq1bBBx/Ayy/DRx8VHWtRu3ZonioeEgW3Jk3U3FQRCgIR2acicx5VNjM46qhwO+WUos99802oaf36/W/Ll8MXXxTdv0GD0kOiXbvwvBRSEIgIsP94hg0bwmNI7qWsJalTBzp0CLeSbNsWBt8VD4l168KSoF9/XXT/pk0Lm5mKh0SbNuGMI52oj0BEgGhGOCeDezhjKOlsYv368DvHLxOakQEtWxYNh9atQ7/EMceEn02aVL0+CvURiMgBFSyAU97tVYUZNGsWbv367f/8nj3w8cclh8SLL4bniv+9XKtWCIWCYIgPieKBURX6KhIaBGZ2GvBroAbwoLvfUez58cBEYA+wHRjn7u8msiYRKVmbNiWfEbRpk/xakqlGjfAXf+vWMHjw/s9/8w188kkIhIKf8ff/9S945RX48sv9X1u7Nhx9dNlh0aJFWK86ysBIWBCYWQ1gFvADIA9YamYLin3RP+buv43tPwK4CzgtUTWJSOmmTy95zqPp06OrKRXUqROazTIzy95v584QDiWFxccfh/EUCxeGCf+Kq127MBjKOstIVGAk8oygH7DW3dcBmNlcYCSwLwjcfWvc/ocBVavDQqQaSYU5j6qyunUL+xTKsmPHgQPj5ZcLFxeKN3MmXHFF5deeyCBoCcSNHyQP6F98JzObCFwD1AZOLulAZjYOGAfQprqfp4pEKDtbX/yJVq9euFqpffuy9ysIjPiwOPHExNQUeWexu88CZpnZBcDNwMUl7DMbmA3hqqHkVigiknzlDYzKkMgLoD4CWsc9bhXbVpq5wBkJrEdEREqQyCBYCnQws3ZmVhs4H1gQv4OZxQ8PGQZ8kMB6RKSKyMkJnbMZGeFnTk7UFVVvCWsacvfdZnY58Dzh8tHfuftqM7sNWObuC4DLzez7wC7gK0poFhKR9JLKI5yrK40sFpGUUl1HOEetrJHFVWyQtIhUd9V1hHMqUxCISEop7QpxXTmeOAoCEUkp06eHEc3xNMI5sRQEIpJSsrNh9uzQJ1CwRvLs2eooTqTIB5SJiBSnEc7JpTMCEZFSpMt4Bp0RiIiUIJ3GM+iMQESkBFOmFJ2SG8LjKVOiqSeRFAQiIiVIp/EMCgIRkRKk03gGBYGISAnSaTyDgkBEpATpNJ5BQSAiUors7DDR3d694WdUIZDoy1h1+aiISApLxmWsOiMQEUlhybiMVUEgIpLCknEZq4JARCSFJeMyVgWBiEgKS8ZlrAoCEZEUlozLWHXVkIhIikv0tNw6IxARSXMKAhGRNKcgEBFJcwoCEZE0pyAQEUlz5u5R11AhZrYR2BB1HYeoKfBF1EWkEH0ehfRZFKXPo6hD+Tzaunuzkp6ockFQHZjZMnfvE3UdqUKfRyF9FkXp8ygqUZ+HmoZERNKcgkBEJM0pCKIxO+oCUow+j0L6LIrS51FUQj4P9RGIiKQ5nRGIiKQ5BYGISJpTECSRmbU2s4Vm9q6ZrTazq6KuKWpmVsPM/mlmf426lqiZ2RFmNs/M1pjZe2Z2fNQ1RcnMJsX+P3nHzP5oZnWjrilZzOx3Zva5mb0Tt62xmf3NzD6I/Tyyst5PQZBcu4Fr3b0zcBww0cw6R1xT1K4C3ou6iBTxa+A5d+8E9CCNPxczawlcCfRx965ADeD8aKtKqoeB04ptmwy85O4dgJdijyuFgiCJ3P0Td18eu7+N8D96y2irio6ZtQKGAQ9GXUvUzKwRMBh4CMDdv3X3zdFWFbmaQD0zqwnUBz6OuJ6kcffFwJfFNo8EHondfwQ4o7LeT0EQETPLBHoC/4i2kkjdDVwP7I26kBTQDtgIzIk1lT1oZodFXVRU3P0jYAbwIfAJsMXdX4i2qsgd5e6fxO5/ChxVWQdWEETAzBoAfwKudvetUdcTBTMbDnzu7m9FXUuKqAn0Au5z957A11TiqX9VE2v/HkkIyBbAYWb2o2irSh0ervuvtGv/FQRJZma1CCGQ4+5/jrqeCA0ERphZLjAXONnM/hBtSZHKA/LcveAMcR4hGNLV94H17r7R3XcBfwYGRFxT1D4zs2MAYj8/r6wDKwiSyMyM0Ab8nrvfFXU9UXL3G929lbtnEjoBX3b3tP2Lz90/Bf5jZsfGNp0CvBthSVH7EDjOzOrH/r85hTTuPI9ZAFwcu38x8GRlHVhBkFwDgQsJf/2uiN1Oj7ooSRlXADlmthLIAm6PuJ7IxM6M5gHLgVWE76q0mW7CzP4I/B041szyzOy/gTuAH5jZB4Qzpjsq7f00xYSISHrTGYGISJpTEIiIpDkFgYhImlMQiIikOQWBiEiaUxCIxJjZnrjLeleYWaWN7DWzzPiZJEVSSc2oCxBJITvcPSvqIkSSTWcEIgdgZrlm9iszW2Vmb5rZd2PbM83sZTNbaWYvmVmb2PajzOwvZvZ27FYwNUINM3sgNsf+C2ZWL7b/lbE1Klaa2dyIfk1JYwoCkUL1ijUNnRf33BZ37wbcQ5g1FeA3wCPu3h3IAWbGts8EXnH3HoT5glbHtncAZrl7F2AzMCq2fTLQM3ac8Yn65URKo5HFIjFmtt3dG5SwPRc42d3XxSYN/NTdm5jZF8Ax7r4rtv0Td29qZhuBVu7+TdwxMoG/xRYVwcxuAGq5+8/N7DlgOzAfmO/u2xP8q4oUoTMCkfLxUu5XxDdx9/dQ2Ec3DJhFOHtYGluIRSRpFAQi5XNe3M+/x+6/TuHyidnAktj9l4AJsG9N5kalHdTMMoDW7r4QuAFoBOx3ViKSSPrLQ6RQPTNbEff4OXcvuIT0yNisoN8Ao2PbriCsKHYdYXWxH8e2XwXMjs0YuYcQCp9QshrAH2JhYcBMLVEpyaY+ApEDiPUR9HH3L6KuRSQR1DQkIpLmdEYgIpLmdEYgIpLmFAQiImlOQSAikuYUBCIiaU5BICKS5v4/ZjylCowYTRwAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "code",
"metadata": {
"id": "0DESR45GQ8WS",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 294
},
"outputId": "b89e2407-e493-43ee-fa0b-53d92082c84d"
},
"source": [
"plt.plot(epochs, acc, 'bo', label='Training acc')\n",
"plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
"plt.title('Training and validation accuracy')\n",
"plt.xlabel('Epochs')\n",
"plt.ylabel('Accuracy')\n",
"plt.legend(loc='lower right')\n",
"\n",
"plt.show()"
],
"execution_count": 159,
"outputs": [
{
"output_type": "display_data",
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEWCAYAAAB8LwAVAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXxU1f3/8deHgEQEURaVPeCG+lW2CBWrgqLiUqhWK5ha0PaLIm78qtTWDRf61dZWa10q1l1acKtVi1pFXKpWCQioKIoaMAgW2VcJ8Pn9cW7IJEySATK5k8z7+XjMY+4+n7mB+5lzzr3nmLsjIiJSUYO4AxARkcykBCEiIkkpQYiISFJKECIikpQShIiIJKUEISIiSSlBSMrM7AUzG1bT28bJzIrMbEAajutmtl80/WczuyaVbXfgcwrM7F87GqdIVUzPQdRvZrYmYbYJ8B2wOZo/390n1H5UmcPMioCfu/srNXxcB/Z393k1ta2Z5QFfAo3cfVNNxClSlYZxByDp5e5NS6eruhiaWUNddCRT6N9jZlAVU5Yys35mVmxmvzSzxcCDZranmT1vZkvMbHk03T5hn9fM7OfR9HAz+7eZ3Rpt+6WZnbSD23Y2szfMbLWZvWJmd5nZY5XEnUqMN5rZW9Hx/mVmrRLWn2Nm881sqZldVcX56WNmi80sJ2HZaWY2O5rubWbvmNkKM1tkZnea2S6VHOshM7spYf6KaJ+vzey8CtueYmbvm9kqM/vKzMYmrH4jel9hZmvM7IjSc5uwf18zm2ZmK6P3vqmem+08zy3M7MHoOyw3s2cS1g02s5nRd/jczAZGy8tV55nZ2NK/s5nlRVVtPzOzBcCr0fInor/DyujfyCEJ++9qZr+P/p4ro39ju5rZP83s4grfZ7aZnZbsu0rllCCy2z5AC6ATMILw7+HBaL4jsB64s4r9+wBzgVbAb4H7zcx2YNu/Au8BLYGxwDlVfGYqMZ4NnAvsBewCXA5gZgcD90THbxt9XnuScPd3gbXAsRWO+9doejMwOvo+RwDHARdWETdRDAOjeI4H9gcqtn+sBX4K7AGcAow0sx9G646O3vdw96bu/k6FY7cA/gncEX23PwD/NLOWFb7DNucmierO86OEKstDomPdFsXQG3gEuCL6DkcDRZWdjySOAQ4CTozmXyCcp72AGUBileitQC+gL+Hf8RhgC/Aw8JPSjcysG9COcG5ke7i7XlnyIvxHHRBN9wM2ArlVbN8dWJ4w/xqhigpgODAvYV0TwIF9tmdbwsVnE9AkYf1jwGMpfqdkMV6dMH8h8GI0fS0wMWHdbtE5GFDJsW8CHoimmxEu3p0q2fYy4O8J8w7sF00/BNwUTT8A3Jyw3QGJ2yY57u3AbdF0XrRtw4T1w4F/R9PnAO9V2P8dYHh152Z7zjPQhnAh3jPJdveWxlvVv79ofmzp3znhu3WpIoY9om2aExLYeqBbku1ygeWEdh0IieTu2v7/Vh9eKkFktyXuvqF0xsyamNm9UZF9FaFKY4/EapYKFpdOuPu6aLLpdm7bFliWsAzgq8oCTjHGxQnT6xJiapt4bHdfCyyt7LMIpYXTzawxcDoww93nR3EcEFW7LI7i+A2hNFGdcjEA8yt8vz5mNjWq2lkJXJDicUuPPb/CsvmEX8+lKjs35VRznjsQ/mbLk+zaAfg8xXiT2XpuzCzHzG6OqqlWUVYSaRW9cpN9VvRvehLwEzNrAAwllHhkOylBZLeKt7D9AjgQ6OPuu1NWpVFZtVFNWAS0MLMmCcs6VLH9zsS4KPHY0We2rGxjd59DuMCeRPnqJQhVVZ8QfqXuDvx6R2IglKAS/RV4Fujg7s2BPycct7pbDr8mVAkl6ggsTCGuiqo6z18R/mZ7JNnvK2DfSo65llB6LLVPkm0Sv+PZwGBCNVxzQimjNIZvgQ1VfNbDQAGh6m+dV6iOk9QoQUiiZoRi+4qoPvu6dH9g9Iu8EBhrZruY2RHAD9IU45PAqWb2/ahB+Qaq/z/wV+BSwgXyiQpxrALWmFlXYGSKMTwODDezg6MEVTH+ZoRf5xui+vyzE9YtIVTtdKnk2JOBA8zsbDNraGZnAQcDz6cYW8U4kp5nd19EaBu4O2rMbmRmpQnkfuBcMzvOzBqYWbvo/ADMBIZE2+cDZ6QQw3eEUl4TQimtNIYthOq6P5hZ26i0cURU2iNKCFuA36PSww5TgpBEtwO7En6d/Qd4sZY+t4DQ0LuUUO8/iXBhSGaHY3T3j4BRhIv+IkI9dXE1u/2N0HD6qrt/m7D8csLFezVwXxRzKjG8EH2HV4F50XuiC4EbzGw1oc3k8YR91wHjgLcs3D31vQrHXgqcSvj1v5TQaHtqhbhTVd15PgcoIZSi/ktog8Hd3yM0gt8GrARep6xUcw3hF/9y4HrKl8iSeYRQglsIzIniSHQ58AEwDVgG3EL5a9ojwKGENi3ZAXpQTjKOmU0CPnH3tJdgpP4ys58CI9z9+3HHUlepBCGxM7PDzWzfqEpiIKHe+Znq9hOpTFR9dyEwPu5Y6jIlCMkE+xBuwVxDuId/pLu/H2tEUmeZ2YmE9ppvqL4aS6qgKiYREUlKJQgREUmq3nTW16pVK8/Ly4s7DBGROmX69OnfunvrZOvqTYLIy8ujsLAw7jBEROoUM6v49P1WqmISEZGklCBERCQpJQgREUlKCUJERJJSghARkaSUIERE6qgJEyAvDxo0CO8TJlS3x/apN7e5iohkkwkTYMQIWBcNtTV/fpgHKCiomc9QCUJEpA666qqy5FBq3bqwvKYoQYiI1EELFmzf8h2hBCEiUgd1rDhYbTXLd4QShIhIHTRuHDRpUn5ZkyZheU1RghAR2U7pvnsoFQUFMH48dOoEZuF9/Piaa6CGNCcIMxtoZnPNbJ6ZXZlkfSczm2Jms83sNTNrn7BumJl9Fr2GpTNOEZFUld49NH8+uJfdPRRXkigqgi1bwntNJgdI44BBZpYDfAocTxgYfhow1N3nJGzzBPC8uz9sZscC57r7OWbWAigE8gEHpgO93H15ZZ+Xn5/v6s1VRNItLy8khYo6dQoX6brGzKa7e36ydeksQfQG5rn7F+6+EZhIGGs40cHAq9H01IT1JwIvu/uyKCm8DAxMY6wiIimpjbuHMkU6E0Q74KuE+eJoWaJZwOnR9GlAMzNrmeK+mNkIMys0s8IlS5bUWOAiIpWpjbuHMkXcjdSXA8eY2fvAMcBCYHOqO7v7eHfPd/f81q2TDogkIvVIJjQO18bdQ5kinQliIdAhYb59tGwrd//a3U939x7AVdGyFansKyLZJVMah2vj7qFMkc5G6oaERurjCBf3acDZ7v5RwjatgGXuvsXMxgGb3f3aqJF6OtAz2nQGoZF6WWWfp0ZqkfqtvjUOZ4pYGqndfRNwEfAS8DHwuLt/ZGY3mNmgaLN+wFwz+xTYGxgX7bsMuJGQVKYBN1SVHESk/sumxuFMkbYSRG1TCUKkflMJIj3ius1VRKTGZFPjcKZQghCRlMR9B1E2NQ5nCg0YJCLVqo3BaVJRUKCEUJtUghCRatXG4DSSeZQgRKRauoMoOylBiEi1sql7CSmjBCEi1dIdRNlJCUJEqqU7iLKT7mISkZToDqLsoxKESIaL+/kDyV4qQYhksEx5/kCyk0oQIhlMzx9InJQgRDKYnj+QOClBiGQwPX8gcVIbhEgGGzeufBsE6PmDbLNhAyxaVP719dfl5/fbD556quY/WwlCJIOVNkRfdVWoVurYMSQHNVDXfWvXVn3RL51fvnzbfXNyYJ99oG3bcGfbYYelJ0YNGCQiUkPcYfXq6i/6ixbBqlXb7r/LLmUX/jZtwivZdKtW4bbnmlDVgEEqQYhIvbR5M2zcCCUl4ZWO6cRkUPpau3bbWHJzyy7uhx4KJ5yQ/MLfokV4Uj1TKEGIVGLCBFXtQPhVvGIFfPVV+AX83XewaVO8r9ILdFUX8HRXjpiF9qDSi3uvXpX/4m/ePLMu/KlSghBJIpseUFu3DoqLQyL86qvwSpz+6itYs2bnPqNBA2jYcMdfu+wSLsal8zk5YVmjRmXvFaerWrcz06XzOTk1c/4zmdogRJLIywtJoaJOnaCoqLaj2XElJeFXf7KLfun80qXb7rf33tChQ3h17Fg23a5d+Qt1Kq+cnJqrL5eapzYIke1UFx5Q27IF/vvf5Bf90unFi8N2iZo3L7vo9+mzbRJo3x4aN47nO0lmUYIQSaJjx+QliNp8QG3jxhDDl1+G14IF5RNAcXHYJlFubtmF/oQTyqYTk0CzZrX3HaRuU4IQSaI2HlDbsiX8wv/ii7IkkDhdXFy+oTUnJzR6duwIvXvDj35U/pd/hw7h9se62BgqmUkJQiSJmnpAbcWKbS/8pdNFReGOoERt20LnznDMMdClS5gufbVtG+r0RWqLGqlFdsKGDWXVQMmSwIoV5bffY4+yC37FBJCXF6qIRGqTGqlFdtCWLbBwYfIqoC+/DOsSNW4cLvSdO8MRR5RPAJ07w557xvI1RHaIEoRIgi1b4IMPYOrU8Hr9dVi5smy9WbjLp3NnOP748hf/Ll1CNwm6pVPqCyUIyWru8PHHZQnhtdfKngvYd18480zIzy9LAB07hoekRLKBEoRkFXeYNw9efbUsIXzzTVjXsSP84AfQv394degQa6gisVOCkIxT030gFRWVJYSpU8vaDdq2hQEDyhJC5866RVQkkRKEZJSa6AOpuLgsGUydWtY1RuvWZcmgf3844AAlBJGq6DZXySg70gfSN9+UTwiffRaW77kn9OsXksGxx8LBByshiFSk21ylzkilD6Rvvw13F02dGqqOPv44LN99dzj6aBg5MiSFww7THUUiO0MJQjJKZX0gtWoFl10WksLs2WHZbrvBUUfB8OEhIfTooSeNRWpSWn9fmdlAM5trZvPM7Mok6zua2VQze9/MZpvZydHyPDNbb2Yzo9ef0xmnZI5x40KfRxUtWQL33hvaEW66Cd56K4zV+8ILMGYMHH64koNITUvbfykzywHuAo4HioFpZvasu89J2Oxq4HF3v8fMDgYmA3nRus/dvXu64pPMU9ozaZs28PnnYbpxYzj5ZLj0Uvje99QNtUhtSudvrt7APHf/AsDMJgKDgcQE4cDu0XRz4Os0xiMZ6ptvQungnntC76YHHgh33QXnnKOuqUXilM4E0Q74KmG+GOhTYZuxwL/M7GJgN2BAwrrOZvY+sAq42t3frPgBZjYCGAHQsTY76pcaMX063HEHTJwYSg8nnRRKCscfr8ZlkUwQ93/DocBD7t4eOBl41MwaAIuAju7eA/h/wF/NbPeKO7v7eHfPd/f81q1b12rgsmNKSuDxx+H73w9dWDz9dHjOYe5cmDwZTjxRyUEkU6SzBLEQSOysoH20LNHPgIEA7v6OmeUCrdz9v8B30fLpZvY5cACgBx3qqG+/hfvug7vvDg+ydekCt90G554bhsAUkcyTzgQxDdjfzDoTEsMQ4OwK2ywAjgMeMrODgFxgiZm1Bpa5+2Yz6wLsD3yRxlglTWbPDtVIEyaEsRMGDAhJ4uSTwwhpIpK50pYg3H2TmV0EvATkAA+4+0dmdgNQ6O7PAr8A7jOz0YQG6+Hu7mZ2NHCDmZUAW4AL3H1ZumKVmrV5Mzz7LPzxj+GBtl13hWHD4OKL4ZBD4o5ORFKlrjakxixfDvffD3feGR5269gRLroIfvYzaNEi7uhEJBl1tSFpNWcO/OlP8MgjoZO9Y46BP/wBBg3Sw2sidZn++8oO2bIl3HV0xx3w8svhAbaCglCN1F2PN4rUC0oQsl1WrYIHHwwlhs8/h3btQvcYI0aE/pJEpP5QgpCUfPppaFt48EFYswb69g2J4fTToVGjuKMTkXRQgpBKucO//hWqkSZPDolgyJBQjXT44XFHJyLppgQh21izJjQ4/+lP8MknsPfeMHYsnH8+7LNP3NGJSG1RgpCttmwJSeG662DlytAVxqOPwplnqhdVkWykBCFA6P5i+HCYMgUGDoRrrw3da2uITpHspQQhTJwYhuksKYHx4+HnP1diEJH4e3OVGC1fDmefDUOHQteucP314c6knBzIywv9J4lI9lIJIktNmRKqlBYvhhtvDN1ijBwZnoSG0FXGiBFhuqAgtjBFJEYqQWSZ9eth9OjQq+puu8E778DVV4c2h9LkUGrdOrjqqnjiFJH4qQSRRd5/H37yk9B30kUXwS23QJMmYd2CBcn3qWy5iNR/KkFkgc2b4eaboU+f0O7w4ovhdtbS5AChiikZjeQqkr2UIOq5L7+Efv3gV7+CwYPhgw/CsJ4VjRtXPmFAmB83rlbCFJEMpARRT7mHfpMOOyyM6vbII2Es6JYtk29fUBBuce3UKdzi2qlTmFcDtUj2UhtEPbRkSegW4+9/D2MzPPxwuOBXp6BACUFEyqgEUc/8859w6KHh/dZb4dVXU0sOIiIVKUHUE2vXhucYTj0V9toLCgvhF7+ABvoLi8gO0uWjHnj33TCK2733whVXwLRpoRQhIrIzlCDqsJKS0PPqkUfCxo0wdSr89rfqeVVEaoYaqeuouXPhnHNCaeGnPw2D+jRvHndUIlKfVFuCMLMfmJlKGhnCHe6+G3r0CGNCP/FEuEtJyUFEaloqF/6zgM/M7Ldm1jXdAUnlFi2Ck0+GUaPg6KPDQ29nnBF3VCJSX1WbINz9J0AP4HPgITN7x8xGmFmztEcnWz31FPzP/8Drr8Odd8ILL0DbtnFHJSL1WUpVR+6+CngSmAi0AU4DZpjZxWmMTQhDfw4bFkoKXbrAjBmhBKEBfUQk3VJpgxhkZn8HXgMaAb3d/SSgG/CL9IaX3d54A7p1g8ceg2uugbffDgP7iIjUhlTuYvoRcJu7v5G40N3XmdnP0hNWdvvuu5AQbr0V9t0X3norjA8tIlKbUkkQY4FFpTNmtiuwt7sXufuUdAWWrT74IIzZMHt2GNHt97+Hpk3jjkpEslEqbRBPAFsS5jdHy6QGuYdkkJ8fhgF97rnwZLSSg4jEJZUE0dDdN5bORNO7pC+k7HTddXD55eE21g8/DH0qiYjEKZUEscTMBpXOmNlg4Nv0hZR9Jk2CG2+Ec8+Fp5+G1q3jjkhEJLU2iAuACWZ2J2DAV8BP0xpVFikshOHD4fvfh3vu0e2rIpI5qk0Q7v458D0zaxrNr0l7VFli4cIwDOjee4eSgzrZE5FMklJnfWZ2CnAIkGvRT1x3vyGNcdV769fDD38Iq1aF5xtUrSQimSaVB+X+TOiP6WJCFdOZQEpjlJnZQDOba2bzzOzKJOs7mtlUM3vfzGab2ckJ634V7TfXzE5M+RvVAe5w3nkwfTpMmKCxG0QkM6XSSN3X3X8KLHf364EjgAOq28nMcoC7gJOAg4GhZnZwhc2uBh539x7AEODuaN+Do/lDgIHA3dHx6oXf/AYmToT/+z8YNKj67UVE4pBKgtgQva8zs7ZACaE/pur0Bua5+xfRrbETgcEVtnFg92i6OfB1ND0YmOju37n7l8C86Hh13tNPw9VXh4fhxoyJOxoRkcqlkiCeM7M9gN8BM4Ai4K8p7NeOcMdTqeJoWaKxwE/MrBiYTKjGSnVfol5lC82scMmSJSmEFK+ZM8MgP336wH336Y4lEclsVSaIaKCgKe6+wt2fIrQ9dHX3a2vo84cCD7l7e+Bk4NHtGZzI3ce7e76757fO8Fbeb74J1UktWsAzz0BubtwRiYhUrcq7mNx9i5ndRRgPAnf/DvguxWMvBDokzLePliX6GaGNAXd/x8xygVYp7ltnbNgAp50GS5fCv/8N++wTd0QiItVL5df6FDP7kdl2V4hMA/Y3s85mtguh0fnZCtssAI4DMLODgFxgSbTdEDNrbGadgf2B97bz8zOCO5x/PrzzThgatEePuCMSEUlNKs9BnA/8P2CTmW0g3Orq7r57VTu5+yYzuwh4CcgBHnD3j8zsBqDQ3Z8ljCdxn5mNJjRYD3d3Bz4ys8eBOcAmYJS7b97B7xirW2+FRx6B66/X8KAiUrdYuB7Xffn5+V5YWBh3GOU8/3xod/jxj+Fvf1OjtIhkHjOb7u75ydZVW4Iws6OTLa84gJCU9+GHMHQo9OwJDzyg5CAidU8qVUxXJEznEp5HmA4cm5aI6oElS+AHP4BmzeAf/4AmTeKOSERk+6XSWd8PEufNrANwe9oiquM2bgxtDYsXw+uvQ7ttnt4QEakbUn7mIEExcFBNB1IfuMOoUfDGG6FaqXeKz35PmAB5edCgQXifMCGdUYqIpCaVNog/Ee4wgpBQuhOeqJYK7rgD/vIXuOqq0P6QigkTwtjT69aF+fnzwzxAQUF64hQRSUW1dzGZ2bCE2U1Akbu/ldaodkDcdzG99FIYLnTwYHjyyVAaSEVeXkgKFXXqBEVFNRmhiMi2duouJuBJYEPpcwhmlmNmTdx9XU0GWZd98gmcdVbotvuRR1JPDgALFmzfchGR2pLSk9TArgnzuwKvpCecumfZsnDHUuPG4Y6lpk23b/+OHbdvuYhIbUklQeQmDjMaTevGTaCkJDwEt2AB/P3voVpoe40bt+1tsE2ahOUiInFKJUGsNbOepTNm1gtYn76Q6o7Ro2HKFBg/Hvr23bFjFBSE/Tt1Cg/TdeoU5tVALSJxS6UN4jLgCTP7mtAP0z6EIUiz2j33wF13wRVXwLBh1W9flYICJQQRyTypPCg3zcy6AgdGi+a6e0l6w8psr74KF18Mp54ahg0VEamPqq1iMrNRwG7u/qG7fwg0NbML0x9aZpo3Lzwp3bVreIYhp96MlC0iUl4qbRD/6+4rSmfcfTnwv+kLKXOtXBnuWGrQAJ59FnavssNzEZG6LZU2iBwzs2icBswsB9glvWFlnk2bYMiQUIJ45RXo0iXuiERE0iuVBPEiMMnM7o3mzwdeSF9ImWnMGHjxxXCH0THHxB2NiEj6pZIgfgmMAC6I5mcT7mTKGvffD7fdBpdeCv+blZVrIpKNqm2DcPctwLtAEWEsiGOBj9MbVuZ44w0YORJOOCEMHyoiki0qLUGY2QHA0Oj1LTAJwN37105o8fvyS/jRj0J7w6RJ0DCV8paISD1R1SXvE+BN4FR3nwdgZqNrJaoMsHp1GE9682Z47jnYY4+4IxIRqV1VVTGdDiwCpprZfWZ2HOFJ6npv8+bwZPPHH8Pjj8P++8cdkYhI7as0Qbj7M+4+BOgKTCV0ubGXmd1jZifUVoBxuOqqUGr44x9hwIC4oxERiUcqjdRr3f2v0djU7YH3CXc21UuPPgq33BIapkeNijsaEZH4bNeY1O6+3N3Hu/tx6QooTv/5D/z859C/fyg9iIhks+1KEPXZggXwwx9Chw7wxBPQqFHcEYmIxEs3bgJr14axpNevh6lToWXLuCMSEYlf1ieILVvCeA6zZ8Pzz8NBB8UdkYhIZsj6KqZ58+Dll8NT0iedFHc0IiKZI+tLEAccEJ53aNMm7khERDJL1icIgLZt445ARCTzZH0Vk4iIJKcEISIiSSlBiIhIUkoQIiKSlBKEiIgkldYEYWYDzWyumc0zsyuTrL/NzGZGr0/NbEXCus0J655NZ5wiIrKttN3mamY5wF3A8UAxMM3MnnX3OaXbuPvohO0vBnokHGK9u3dPV3wiIlK1dJYgegPz3P0Ld98ITAQGV7H9UOBvaYxHRES2QzoTRDvgq4T54mjZNsysE9AZeDVhca6ZFZrZf8zsh5XsNyLapnDJkiU1FbeIiJA5jdRDgCfdfXPCsk7ung+cDdxuZvtW3CkamyLf3fNbt25dW7GKiGSFdCaIhUCHhPn20bJkhlChesndF0bvXwCvUb59QkRE0iydCWIasL+ZdTazXQhJYJu7kcysK7An8E7Csj3NrHE03Qo4EphTcV8REUmftN3F5O6bzOwi4CUgB3jA3T8ysxuAQncvTRZDgInu7gm7HwTca2ZbCEns5sS7n0REJP2s/HW57srPz/fCwsK4wxARqVPMbHrU3ruNTGmkFhGRDKMEISIiSSlBiIhIUkoQIiKSlBKEiIgkpQQhIiJJKUGIiEhSShAiIpKUEoSIiCSlBCEiIkkpQYiISFJKECIikpQShIiIJKUEISIiSSlBiIhIUkoQIiKSlBKEiIgkpQQhIiJJKUGIiEhSShAiIpKUEoSIiCSlBCEiIkkpQYiISFJKECIikpQShIiIJNUw7gBEpO4rKSmhuLiYDRs2xB2KVCI3N5f27dvTqFGjlPdRghCRnVZcXEyzZs3Iy8vDzOIORypwd5YuXUpxcTGdO3dOeT9VMYnITtuwYQMtW7ZUcshQZkbLli23u4SnBCEiNULJIbPtyN9HCUJERJJSghCRWjdhAuTlQYMG4X3ChJ073tKlS+nevTvdu3dnn332oV27dlvnN27cWOW+hYWFXHLJJdV+Rt++fXcuyDpIjdQiUqsmTIARI2DdujA/f36YBygo2LFjtmzZkpkzZwIwduxYmjZtyuWXX751/aZNm2jYMPnlLj8/n/z8/Go/4+23396x4OowlSBEpFZddVVZcii1bl1YXpOGDx/OBRdcQJ8+fRgzZgzvvfceRxxxBD169KBv377MnTsXgNdee41TTz0VCMnlvPPOo1+/fnTp0oU77rhj6/GaNm26dft+/fpxxhln0LVrVwoKCnB3ACZPnkzXrl3p1asXl1xyydbjJioqKuKoo46iZ8+e9OzZs1ziueWWWzj00EPp1q0bV155JQDz5s1jwIABdOvWjZ49e/L555/X7ImqgkoQIlKrFizYvuU7o7i4mLfffpucnBxWrVrFm2++ScOGDXnllVf49a9/zVNPPbXNPp988glTp05l9erVHHjggYwcOXKbZwfef/99PvroI9q2bcuRRx7JW2+9RX5+Pueffz5vvPEGnTt3ZujQoUlj2muvvXj55ZfJzc3ls88+Y+jQoRQWFvLCCy/wj3/8g3fffZcmTZqwbNkyAAoKCrjyyis57bTT2LBhA1u2bKn5E1UJJQgRqVUdO4ZqpWTLa9qZZ55JTk4OACtXrmTYsGF89tlnmBklJSVJ9znllFNo3LgxjRs3Zq+99uKbb76hffv25bbp3bv31mXdu3enqKiIpk2b0qVLl63PGVCwkwMAAAyTSURBVAwdOpTx48dvc/ySkhIuuugiZs6cSU5ODp9++ikAr7zyCueeey5NmjQBoEWLFqxevZqFCxdy2mmnAeFht9qU1iomMxtoZnPNbJ6ZXZlk/W1mNjN6fWpmKxLWDTOzz6LXsHTGKSK1Z9w4iK6BWzVpEpbXtN12223r9DXXXEP//v358MMPee655yp9JqBx48Zbp3Nycti0adMObVOZ2267jb333ptZs2ZRWFhYbSN6nNKWIMwsB7gLOAk4GBhqZgcnbuPuo929u7t3B/4EPB3t2wK4DugD9AauM7M90xWriNSeggIYPx46dQKz8D5+/I43UKdq5cqVtGvXDoCHHnqoxo9/4IEH8sUXX1BUVATApEmTKo2jTZs2NGjQgEcffZTNmzcDcPzxx/Pggw+yLmqgWbZsGc2aNaN9+/Y888wzAHz33Xdb19eGdJYgegPz3P0Ld98ITAQGV7H9UOBv0fSJwMvuvszdlwMvAwPTGKuI1KKCAigqgi1bwnu6kwPAmDFj+NWvfkWPHj226xd/qnbddVfuvvtuBg4cSK9evWjWrBnNmzffZrsLL7yQhx9+mG7duvHJJ59sLeUMHDiQQYMGkZ+fT/fu3bn11lsBePTRR7njjjs47LDD6Nu3L4sXL67x2Ctjpa3vNX5gszOAge7+82j+HKCPu1+UZNtOwH+A9u6+2cwuB3Ld/aZo/TXAene/tcJ+I4ARAB07duw1P1nFpoik3ccff8xBBx0UdxixW7NmDU2bNsXdGTVqFPvvvz+jR4+OO6ytkv2dzGy6uye9zzdTbnMdAjzp7pu3Zyd3H+/u+e6e37p16zSFJiKSmvvuu4/u3btzyCGHsHLlSs4///y4Q9op6byLaSHQIWG+fbQsmSHAqAr79quw72s1GJuISI0bPXp0RpUYdlY6SxDTgP3NrLOZ7UJIAs9W3MjMugJ7Au8kLH4JOMHM9owap0+IlomISC1JWwnC3TeZ2UWEC3sO8IC7f2RmNwCF7l6aLIYAEz2hMcTdl5nZjYQkA3CDuy9LV6wiIrKttD4o5+6TgckVll1bYX5sJfs+ADyQtuBERKRKmdJILSIiGUYJQkTqvP79+/PSS+WbKW+//XZGjhxZ6T79+vWjsLAQgJNPPpkVK1Zss83YsWO3Po9QmWeeeYY5c+Zsnb/22mt55ZVXtif8jKUEISJ13tChQ5k4cWK5ZRMnTqy0w7yKJk+ezB577LFDn10xQdxwww0MGDBgh46VadRZn4jUqMsug2hohhrTvTvcfnvl68844wyuvvpqNm7cyC677EJRURFff/01Rx11FCNHjmTatGmsX7+eM844g+uvv36b/fPy8igsLKRVq1aMGzeOhx9+mL322osOHTrQq1cvIDzjMH78eDZu3Mh+++3Ho48+ysyZM3n22Wd5/fXXuemmm3jqqae48cYbOfXUUznjjDOYMmUKl19+OZs2beLwww/nnnvuoXHjxuTl5TFs2DCee+45SkpKeOKJJ+jatWu5mIqKijjnnHNYu3YtAHfeeefWQYtuueUWHnvsMRo0aMBJJ53EzTffzLx587jgggtYsmQJOTk5PPHEE+y77747dd5VghCROq9Fixb07t2bF154AQilhx//+MeYGePGjaOwsJDZs2fz+uuvM3v27EqPM336dCZOnMjMmTOZPHky06ZN27ru9NNPZ9q0acyaNYuDDjqI+++/n759+zJo0CB+97vfMXPmzHIX5A0bNjB8+HAmTZrEBx98wKZNm7jnnnu2rm/VqhUzZsxg5MiRSauxSrsFnzFjBpMmTdo66l1it+CzZs1izJgxQOgWfNSoUcyaNYu3336bNm3a7NxJRSUIEalhVf3ST6fSaqbBgwczceJE7r//fgAef/xxxo8fz6ZNm1i0aBFz5szhsMMOS3qMN998k9NOO21rl9uDBg3auu7DDz/k6quvZsWKFaxZs4YTTzyxynjmzp1L586dOeCAAwAYNmwYd911F5dddhkQEg5Ar169ePrpp7fZPxO6Bc/6EkRNj40rIvEYPHgwU6ZMYcaMGaxbt45evXrx5ZdfcuuttzJlyhRmz57NKaecUmk339UZPnw4d955Jx988AHXXXfdDh+nVGmX4ZV1F54J3YJndYIoHRt3/nxwLxsbV0lCpO5p2rQp/fv357zzztvaOL1q1Sp22203mjdvzjfffLO1CqoyRx99NM888wzr169n9erVPPfcc1vXrV69mjZt2lBSUsKEhItEs2bNWL169TbHOvDAAykqKmLevHlA6JX1mGOOSfn7ZEK34FmdIGprbFwRqR1Dhw5l1qxZWxNEt27d6NGjB127duXss8/myCOPrHL/nj17ctZZZ9GtWzdOOukkDj/88K3rbrzxRvr06cORRx5ZrkF5yJAh/O53v6NHjx7lxovOzc3lwQcf5Mwzz+TQQw+lQYMGXHDBBSl/l0zoFjxt3X3Xtvz8fC+9pzlVDRqEkkNFZqGfehFJjbr7rhvqanffsahsDNx0jI0rIlLXZHWCqM2xcUVE6pqsThBxjY0rUh/Vl+rq+mpH/j5Z/xxEQYESgsjOys3NZenSpbRs2RIzizscqcDdWbp06XY/H5H1CUJEdl779u0pLi5myZIlcYcilcjNzaV9+/bbtY8ShIjstEaNGtG5c+e4w5AaltVtECIiUjklCBERSUoJQkREkqo3T1Kb2RJgftxx7KRWwLdxB5FBdD7K0/koo3NR3s6cj07u3jrZinqTIOoDMyus7JH3bKTzUZ7ORxmdi/LSdT5UxSQiIkkpQYiISFJKEJllfNwBZBidj/J0PsroXJSXlvOhNggREUlKJQgREUlKCUJERJJSgsgAZtbBzKaa2Rwz+8jMLo07priZWY6ZvW9mz8cdS9zMbA8ze9LMPjGzj83siLhjipOZjY7+n3xoZn8zs+3rorSOM7MHzOy/ZvZhwrIWZvaymX0Wve9ZE5+lBJEZNgG/cPeDge8Bo8zs4JhjitulwMdxB5Eh/gi86O5dgW5k8Xkxs3bAJUC+u/8PkAMMiTeqWvcQMLDCsiuBKe6+PzAlmt9pShAZwN0XufuMaHo14QLQLt6o4mNm7YFTgL/EHUvczKw5cDRwP4C7b3T3FfFGFbuGwK5m1hBoAnwdczy1yt3fAJZVWDwYeDiafhj4YU18lhJEhjGzPKAH8G68kcTqdmAMsCXuQDJAZ2AJ8GBU5fYXM9st7qDi4u4LgVuBBcAiYKW7/yveqDLC3u6+KJpeDOxdEwdVgsggZtYUeAq4zN1XxR1PHMzsVOC/7j497lgyREOgJ3CPu/cA1lJD1Qd1UVS3PpiQONsCu5nZT+KNKrN4eHahRp5fUILIEGbWiJAcJrj703HHE6MjgUFmVgRMBI41s8fiDSlWxUCxu5eWKJ8kJIxsNQD40t2XuHsJ8DTQN+aYMsE3ZtYGIHr/b00cVAkiA1gYxPd+4GN3/0Pc8cTJ3X/l7u3dPY/Q+Piqu2ftL0R3Xwx8ZWYHRouOA+bEGFLcFgDfM7Mm0f+b48jiRvsEzwLDoulhwD9q4qBKEJnhSOAcwq/lmdHr5LiDkoxxMTDBzGYD3YHfxBxPbKKS1JPADOADwjUsq7rdMLO/Ae8AB5pZsZn9DLgZON7MPiOUsm6ukc9SVxsiIpKMShAiIpKUEoSIiCSlBCEiIkkpQYiISFJKECIikpQShEg1zGxzwu3HM82sxp5kNrO8xF45RTJJw7gDEKkD1rt797iDEKltKkGI7CAzKzKz35rZB2b2npntFy3PM7NXzWy2mU0xs47R8r3N7O9mNit6lXYRkWNm90VjHPzLzHaNtr8kGiNktplNjOlrShZTghCp3q4VqpjOSli30t0PBe4k9EIL8CfgYXc/DJgA3BEtvwN43d27EfpT+ihavj9wl7sfAqwAfhQtvxLoER3ngnR9OZHK6ElqkWqY2Rp3b5pkeRFwrLt/EXW2uNjdW5rZt0Abdy+Jli9y91ZmtgRo7+7fJRwjD3g5GugFM/sl0MjdbzKzF4E1wDPAM+6+Js1fVaQclSBEdo5XMr09vkuY3kxZ2+ApwF2E0sa0aIAckVqjBCGyc85KeH8nmn6bsmEwC4A3o+kpwEjYOuZ288oOamYNgA7uPhX4JdAc2KYUI5JO+kUiUr1dzWxmwvyL7l56q+ueUS+r3wFDo2UXE0aAu4IwGty50fJLgfFR75ubCcliEcnlAI9FScSAOzTUqNQ2tUGI7KCoDSLf3b+NOxaRdFAVk4iIJKUShIiIJKUShIiIJKUEISIiSSlBiIhIUkoQIiKSlBKEiIgk9f8BZcQHYYGneCkAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"tags": [],
"needs_background": "light"
}
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zfavK608SbG4",
"colab_type": "text"
},
"source": [
"In this plot, the dots represent the training loss and accuracy, and the solid lines are the validation loss and accuracy.\n",
"\n",
"Notice the training loss decreases with each epoch and the training accuracy increases with each epoch. This is expected when using a gradient descent optimization—it should minimize the desired quantity on every iteration.\n",
"\n",
"This isn't the case for the validation loss and accuracy—they seem to peak before the training accuracy. This is an example of overfitting: the model performs better on the training data than it does on data it has never seen before. After this point, the model over-optimizes and learns representations specific to the training data that do not generalize to test data.\n",
"\n",
"For this particular case, you could prevent overfitting by simply stopping the training when the validation accuracy is no longer increasing. One way to do so is to use the `EarlyStopping` callback."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sJZbc_SmARUq",
"colab_type": "text"
},
"source": [
"---"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "wdm8YKwJf0pF",
"colab_type": "text"
},
"source": [
"## Clustering in Tensorflow\n",
"\n",
"In machine learning, we often group examples as a first step to understand a subject (data set) in a machine learning system. Grouping unlabeled examples is called clustering.\n",
"\n",
"As the examples are **unlabeled**, clustering relies on **unsupervised machine learning**. If the examples are **labeled**, then clustering becomes **classification**.\n",
"\n",
"Broadly, there are two main kinds of Clustering Algorithm\n",
"- K-Means Clustering\n",
"- Hierachical Clustering\n",
"\n",
"Clustering has a myriad of uses in a variety of industries. Some common applications for clustering include the following:\n",
"\n",
" - market segmentation\n",
" - social network analysis\n",
" - search result grouping\n",
" - medical imaging\n",
" - image segmentation\n",
" - anomaly detection\n",
"\n",
"### **Hidden Markov Model**\n",
"\n",
"\n",
"\"The Hidden Markov Model is a finite set of states, each of which is associated with a (generally multidimensional) probability distribution. Transitions among the states are governed by a set of probabilities called transition probabilities.\" (http://jedlik.phy.bme.hu/~gerjanos/HMM/node4.html)\n",
"\n",
"A hidden markov model works with probabilities to predict future events or states.\n",
"\n",
"**States:** In each markov model we have a finite set of states. These states could be something like \"warm\" and \"cold\" or \"high\" and \"low\" or even \"red\", \"green\" and \"blue\". These states are \"hidden\" within the model, which means we do not direcly observe them.\n",
"\n",
"**Observations:** Each state has a particular outcome or observation associated with it based on a probability distribution. An example of this is the following: \n",
"> *On a hot day Tim has a 80% chance of being happy and a 20% chance of being sad.*\n",
"\n",
"**Transitions:** Each state will have a probability defining the likelyhood of transitioning to a different state. An example is the following: \n",
"> *a cold day has a 30% chance of being followed by a hot day and a 70% chance of being follwed by another cold day.*\n",
"\n",
"To create a hidden markov model we need.\n",
"- States\n",
"- Observation Distribution\n",
"- Transition Distribution\n",
"\n",
"Read more about it from the links given below:\n",
"\n",
"**Links**:\n",
"- https://medium.com/@kangeugine/hidden-markov-model-7681c22f5b9\n",
"- https://en.wikipedia.org/wiki/Hidden_Markov_model\n",
"- https://www.tensorflow.org/probability/\n",
"- https://www.tensorflow.org/probability/api_docs/python/tfp/distributions/HiddenMarkovModel\n",
"- https://www.tensorflow.org/probability/api_docs/python/tfp/edward2/HiddenMarkovModel\n"
]
},
{
"cell_type": "code",
"metadata": {
"id": "y2VCgiG9OtYb",
"colab_type": "code",
"colab": {}
},
"source": [
"from IPython.display import clear_output\n",
"import tensorflow_probability as tfp\n",
"import tensorflow as tf"
],
"execution_count": 162,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "58I0VyKMsVNk",
"colab_type": "text"
},
"source": [
"### A Weather Model\n",
"\n",
"\n",
"We will model a simple weather system and try to predict the average temperature on each day given the following information.\n",
"1. Cold days are encoded by a 0 and hot days are encoded by a 1.\n",
"2. The first day in our sequence has an 80% chance of being cold.\n",
"3. A cold day has a 30% chance of being followed by a hot day.\n",
"4. A hot day has a 20% chance of being followed by a cold day.\n",
"5. On each day the temperature is\n",
" normally distributed with mean and standard deviation 0 and 5 on\n",
" a cold day and mean and standard deviation 15 and 10 on a hot day.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "0sfPi_HYyxxt",
"colab_type": "text"
},
"source": [
"#### Define the distribution variables of the model"
]
},
{
"cell_type": "code",
"metadata": {
"id": "5p62mw_ilpLk",
"colab_type": "code",
"colab": {}
},
"source": [
"tfd = tfp.distributions\n",
"\n",
"# probs[0] : probability of day being cold\n",
"# probs[1] : probability of day being hot\n",
"\n",
"# first day : initial_distribution\n",
"# 80% chance of cold, 20% chance of hot\n",
"init_dist = tfd.Categorical(probs=[0.8, 0.2])\n",
"\n",
"# after first day : transitiion_distribution\n",
"# 70% chance of cold, 30% chance of hot\n",
"# 20% chance of cold, 80% chance of hot \n",
"trans_dist = tfd.Categorical(probs=[[0.7, 0.3],\n",
" [0.2, 0.8]])\n",
"\n",
"# Each day Temperature Distribution : observation_distribution\n",
"# Cold day : Mean = 0, Std. Deviation = 5\n",
"# Hot Day : Mean = 15, Std. Deviation = 10\n",
"# For any single day : Range of mean temp. = [0, 15]\n",
"# For any single day : Range of std. dev temp. = [5, 10]\n",
"obs_dist = tfd.Normal(loc=[0., 15.], scale = [5., 10.]) "
],
"execution_count": 163,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "FSvWongiy-Wt",
"colab_type": "text"
},
"source": [
"#### Create our `HiddenMarkovModel`"
]
},
{
"cell_type": "code",
"metadata": {
"id": "rspJP8eWzpct",
"colab_type": "code",
"colab": {}
},
"source": [
"# combine distributions into a single week long\n",
"hmm_model = tfd.HiddenMarkovModel(\n",
" initial_distribution = init_dist,\n",
" transition_distribution = trans_dist,\n",
" observation_distribution = obs_dist,\n",
" # num_steps: number of days in a week\n",
" num_steps = 7\n",
" )\n",
"\n",
"clear_output()"
],
"execution_count": 164,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "ep6l8huz20Vw",
"colab_type": "text"
},
"source": [
"#### Get Average Temperature Values For a Week "
]
},
{
"cell_type": "code",
"metadata": {
"id": "6fA53GyV2El3",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 33
},
"outputId": "9af16ccd-d553-4991-fcb8-30a150e762a9"
},
"source": [
"mean = hmm_model.mean()\n",
"\n",
"print(mean.numpy())"
],
"execution_count": 165,
"outputs": [
{
"output_type": "stream",
"text": [
"[3. 6. 7.5 8.249999 8.625001 8.812501 8.90625 ]\n"
],
"name": "stdout"
}
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "GT5TapMY5C8-",
"colab_type": "text"
},
"source": [
"---"
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment