Skip to content

Instantly share code, notes, and snippets.

@elijah123815

elijah123815/8.5.1BachNorm_v2.ipynb

Created October 27, 2020 18:03
Show Gist options
  • Save elijah123815/ced3975a687c102559b6052feeef4be7 to your computer and use it in GitHub Desktop.
Save elijah123815/ced3975a687c102559b6052feeef4be7 to your computer and use it in GitHub Desktop.
Download ZIP
Created on Skills Network Labs
Display the source blob
Display the rendered blob
Raw
8.5.1BachNorm_v2.ipynb
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"http://cocl.us/pytorch_link_top\">\n",
" <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/Pytochtop.png\" width=\"750\" alt=\"IBM Product \" />\n",
"</a> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/cc-logo-square.png\" width=\"200\" alt=\"cognitiveclass.ai logo\" />\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h1>Batch Normalization with the MNIST Dataset</h1>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h3>Objective for this Notebook<h3> \n",
"<h5> 1. Define Several Neural Networks, Criterion function, Optimizer.</h5>\n",
"<h5> 2. Train Neural Network using Batch Normalization and no Batch Normalization </h5> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Table of Contents</h2>\n",
"In this lab, you will build a Neural Network using Batch Normalization and compare it to a Neural Network that does not use Batch Normalization. You will use the MNIST dataset to test your network. \n",
"\n",
"<ul>\n",
"<li><a href=\"#Train_Func\">Neural Network Module and Training Function</a></li>\n",
"<li><a href=\"#Makeup_Data\">Load Data </a></li>\n",
"<li><a href=\"#NN\">Define Several Neural Networks, Criterion function, Optimizer</a></li>\n",
"<li><a href=\"#Train\">Train Neural Network using Batch Normalization and no Batch Normalization</a></li>\n",
"<li><a href=\"#Result\">Analyze Results</a></li>\n",
"</ul>\n",
"<p>Estimated Time Needed: <strong>25 min</strong></p>\n",
"</div>\n",
"\n",
"<hr>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>Preparation</h2>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We'll need the following libraries: \n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"<torch._C.Generator at 0x7f97df7aab30>"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# These are the libraries will be used for this lab.\n",
"\n",
"# Using the following line code to install the torchvision library\n",
"# !conda install -y torchvision\n",
"\n",
"import torch \n",
"import torch.nn as nn\n",
"import torchvision.transforms as transforms\n",
"import torchvision.datasets as dsets\n",
"import torch.nn.functional as F\n",
"import matplotlib.pylab as plt\n",
"import numpy as np\n",
"torch.manual_seed(0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!--Empty Space for separating topics-->\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2 id=\"Train_Func\">Neural Network Module and Training Function</h2> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define the neural network module or class \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
" Neural Network Module with two hidden layers using Batch Normalization\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"# Define the Neural Network Model using Batch Normalization\n",
"\n",
"class NetBatchNorm(nn.Module):\n",
" \n",
" # Constructor\n",
" def __init__(self, in_size, n_hidden1, n_hidden2, out_size):\n",
" super(NetBatchNorm, self).__init__()\n",
" self.linear1 = nn.Linear(in_size, n_hidden1)\n",
" self.linear2 = nn.Linear(n_hidden1, n_hidden2)\n",
" self.linear3 = nn.Linear(n_hidden2, out_size)\n",
" self.bn1 = nn.BatchNorm1d(n_hidden1)\n",
" self.bn2 = nn.BatchNorm1d(n_hidden2)\n",
" \n",
" # Prediction\n",
" def forward(self, x):\n",
" x = self.bn1(torch.sigmoid(self.linear1(x)))\n",
" x = self.bn2(torch.sigmoid(self.linear2(x)))\n",
" x = self.linear3(x)\n",
" return x\n",
" \n",
" # Activations, to analyze results \n",
" def activation(self, x):\n",
" out = []\n",
" z1 = self.bn1(self.linear1(x))\n",
" out.append(z1.detach().numpy().reshape(-1))\n",
" a1 = torch.sigmoid(z1)\n",
" out.append(a1.detach().numpy().reshape(-1).reshape(-1))\n",
" z2 = self.bn2(self.linear2(a1))\n",
" out.append(z2.detach().numpy().reshape(-1))\n",
" a2 = torch.sigmoid(z2)\n",
" out.append(a2.detach().numpy().reshape(-1))\n",
" return out"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Neural Network Module with two hidden layers with out Batch Normalization\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"# Class Net for Neural Network Model\n",
"\n",
"class Net(nn.Module):\n",
" \n",
" # Constructor\n",
" def __init__(self, in_size, n_hidden1, n_hidden2, out_size):\n",
"\n",
" super(Net, self).__init__()\n",
" self.linear1 = nn.Linear(in_size, n_hidden1)\n",
" self.linear2 = nn.Linear(n_hidden1, n_hidden2)\n",
" self.linear3 = nn.Linear(n_hidden2, out_size)\n",
" \n",
" # Prediction\n",
" def forward(self, x):\n",
" x = torch.sigmoid(self.linear1(x))\n",
" x = torch.sigmoid(self.linear2(x))\n",
" x = self.linear3(x)\n",
" return x\n",
" \n",
" # Activations, to analyze results \n",
" def activation(self, x):\n",
" out = []\n",
" z1 = self.linear1(x)\n",
" out.append(z1.detach().numpy().reshape(-1))\n",
" a1 = torch.sigmoid(z1)\n",
" out.append(a1.detach().numpy().reshape(-1).reshape(-1))\n",
" z2 = self.linear2(a1)\n",
" out.append(z2.detach().numpy().reshape(-1))\n",
" a2 = torch.sigmoid(z2)\n",
" out.append(a2.detach().numpy().reshape(-1))\n",
" return out \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Define a function to train the model. In this case the function returns a Python dictionary to store the training loss and accuracy on the validation data \n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"# Define the function to train model\n",
"\n",
"def train(model, criterion, train_loader, validation_loader, optimizer, epochs=100):\n",
" i = 0\n",
" useful_stuff = {'training_loss':[], 'validation_accuracy':[]} \n",
"\n",
" for epoch in range(epochs):\n",
" for i, (x, y) in enumerate(train_loader):\n",
" model.train()\n",
" optimizer.zero_grad()\n",
" z = model(x.view(-1, 28 * 28))\n",
" loss = criterion(z, y)\n",
" loss.backward()\n",
" optimizer.step()\n",
" useful_stuff['training_loss'].append(loss.data.item())\n",
" \n",
" correct = 0\n",
" for x, y in validation_loader:\n",
" model.eval()\n",
" yhat = model(x.view(-1, 28 * 28))\n",
" _, label = torch.max(yhat, 1)\n",
" correct += (label == y).sum().item()\n",
" \n",
" accuracy = 100 * (correct / len(validation_dataset))\n",
" useful_stuff['validation_accuracy'].append(accuracy)\n",
" \n",
" return useful_stuff"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!--Empty Space for separating topics-->\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2 id=\"Makeup_Data\">Make Some Data</h2> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load the training dataset by setting the parameters <code>train </code> to <code>True</code> and convert it to a tensor by placing a transform object int the argument <code>transform</code>\n"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [],
"source": [
"# load the train dataset\n",
"\n",
"train_dataset = dsets.MNIST(root='./data', train=True, download=True, transform=transforms.ToTensor())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Load the validating dataset by setting the parameters train <code>False</code> and convert it to a tensor by placing a transform object into the argument <code>transform</code>\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"# load the train dataset\n",
"\n",
"validation_dataset = dsets.MNIST(root='./data', train=False, download=True, transform=transforms.ToTensor())"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"create the training-data loader and the validation-data loader object \n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"# Create Data Loader for both train and validating\n",
"\n",
"train_loader = torch.utils.data.DataLoader(dataset=train_dataset, batch_size=2000, shuffle=True)\n",
"validation_loader = torch.utils.data.DataLoader(dataset=validation_dataset, batch_size=5000, shuffle=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a id=\"ref3\"></a>\n",
"\n",
"<h2 align=center>Define Neural Network, Criterion function, Optimizer and Train the Model </h2> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create the criterion function \n"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"# Create the criterion function\n",
"\n",
"criterion = nn.CrossEntropyLoss()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Variables for Neural Network Shape <code> hidden_dim</code> used for number of neurons in both hidden layers.\n"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [],
"source": [
"# Set the parameters\n",
"\n",
"input_dim = 28 * 28\n",
"hidden_dim = 100\n",
"output_dim = 10"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!--Empty Space for separating topics-->\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2 id=\"Train\">Train Neural Network using Batch Normalization and no Batch Normalization </h2> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Train Neural Network using Batch Normalization :\n"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [],
"source": [
"# Create model, optimizer and train the model\n",
"\n",
"model_norm = NetBatchNorm(input_dim, hidden_dim, hidden_dim, output_dim)\n",
"optimizer = torch.optim.Adam(model_norm.parameters(), lr = 0.1)\n",
"training_results_Norm=train(model_norm , criterion, train_loader, validation_loader, optimizer, epochs=5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Train Neural Network with no Batch Normalization:\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [],
"source": [
"# Create model without Batch Normalization, optimizer and train the model\n",
"\n",
"model = Net(input_dim, hidden_dim, hidden_dim, output_dim)\n",
"optimizer = torch.optim.Adam(model.parameters(), lr = 0.1)\n",
"training_results = train(model, criterion, train_loader, validation_loader, optimizer, epochs=5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2 id=\"Result\">Analyze Results</h2> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compare the histograms of the activation for the first layer of the first sample, for both models.\n"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAEHCAYAAAC3Ph1GAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcQklEQVR4nO3de3RU9bn/8fdDiCaKIApiFDTShRckEChEVOQimh8qFWzBG9bYqhHvpx6VVNfvEHuqooee1tXLzyJVsix6QpEWqvacaiQFLEqChotFiloUSg7EqBQrVojP74/ZSUMySSY3km/4vNbKmj179ux5vjPwYfPNnmebuyMiIuHp1tEFiIhIyyjARUQCpQAXEQmUAlxEJFAKcBGRQCnARUQC1T2RjczsaGA+MARw4NvAZqAQSAe2Ape7+8eN7adPnz6enp7e4mJFRA5Fa9eu/dDd+9Zdb4mcB25mBcBKd59vZocBRwD3AR+5+xwzywN6u/usxvYzcuRILy0tbdkIREQOUWa21t1H1l3f5BSKmfUExgK/AHD3L9z9E2AKUBBtVgBMbatiRUSkaYnMgQ8EKoCnzOxNM5tvZkcC/dy9HCC6Pa4d6xQRkToSCfDuwAjg/7n7cODvQF6iL2BmuWZWamalFRUVLSxTRETqSuSXmNuB7e7+enR/MbEA32lmae5ebmZpwK54T3b3ecA8iM2Bt0HN0knt27eP7du38/nnn3d0KSJBSklJoX///iQnJye0fZMB7u7/a2bbzOw0d98MTAT+FP3kAHOi26UtL1u6gu3bt3PUUUeRnp6OmXV0OSJBcXcqKyvZvn07p5xySkLPSeg0QuB2YGF0Bsp7wLeITb8sMrPrgQ+A6S2oWbqQzz//XOEt0kJmxrHHHktzppoTCnB3LwPqncJC7GhcpIbCW6Tlmvv3R9/EFBEJVKJTKCLNlp73Qpvub+ucS9p0f01JT0+ntLSUPn36tGqbeG644QbuuusuBg8ezEMPPcR9990HwNatW5k8eTIbN25sVe2NaclrLFiwgOzsbE444YRGtyktLeUnP/lJW5TZbq677jomT57MtGnTDvgcmqPu+9HS/bRWlw/wtg6R2g52oEjXMX/+/Jrl2gHeWS1YsIAhQ4Y0GuAHw/79++neve1iq/bn0Bx134+W7qe1NIUiXcbWrVs5/fTTueGGGxgyZAgzZszg5Zdf5txzz2XQoEGsWbMGgI8++oipU6cydOhQRo8ezfr16wGorKwkOzub4cOHc9NNN1G7zcQvf/lLsrKyyMzM5KabbqKqqqrBOhYtWsRdd90FwGOPPcbAgQMBePfddxkzZgwA48ePp7S0lLy8PPbu3UtmZiYzZswAoKqqihtvvJEzzzyT7Oxs9u7dW+81rrvuOu644w7OOeccBg4cyOLFi4HYmQz33HMPQ4YMISMjg8LCwrg17t+/n5ycHIYOHcq0adP47LPPAPje977HqFGjGDJkCLm5ubg7ixcvprS0lBkzZpCZmcnevXspKSnhnHPOYdiwYWRlZbFnzx4AduzYwaRJkxg0aBD33ntv3NdOT09n9uzZjBgxgoyMDN5+++1GP5f8/Hxyc3PJzs7m2muvJT8/n5ycHLKzs0lPT2fJkiXce++9ZGRkMGnSJPbt29fgWOqq/hyWLVtGZmYmmZmZnHbaaTVngST6flTvB+DZZ58lIyODIUOGMGvWP7uL9OjRg/vvv59hw4YxevRodu7cGff9aQ4FuHQp77zzDnfeeSfr16/n7bff5plnnmHVqlXMnTuXhx56CIDZs2czfPhw1q9fz0MPPcS1114LwAMPPMCYMWN48803ufTSS/nggw8A2LRpE4WFhbz66quUlZWRlJTEwoULG6xh7NixrFy5EoCVK1dy7LHH8te//pVVq1Zx3nnnHbDtnDlzSE1NpaysrGafW7Zs4dZbb+Wtt97i6KOP5rnnnov7OuXl5axatYrnn3+evLzYd+uWLFlCWVkZ69at4+WXX+aee+6hvLy83nM3b95Mbm4u69evp2fPnvzsZz8D4LbbbqOkpISNGzeyd+9enn/+eaZNm8bIkSNZuHBhzfivuOIKHnvssZrXSU1NBaCsrIzCwkI2bNhAYWEh27Zti1t7nz59eOONN7j55puZO3duo58LwNq1a1m6dCnPPPMMEPvH8IUXXmDp0qVcc801TJgwgQ0bNpCamsoLL7zQ4Fgacumll1JWVkZZWRnDhg3j7rvvTvj9qB47xP4BmzVrFq+88gplZWWUlJTwm9/8BoC///3vjB49mnXr1jF27FieeOKJButJlAJcupRTTjmFjIwMunXrxplnnsnEiRMxMzIyMti6dSsAq1at4pvf/CYA559/PpWVlezevZsVK1ZwzTXXAHDJJZfQu3dvAIqKili7di2jRo0iMzOToqIi3nvvvQZrOP744/n000/Zs2cP27Zt4+qrr2bFihWsXLmyXoA3NIbMzEwAvvrVr9bUXdfUqVPp1q0bgwcPrjmaW7VqFVdddRVJSUn069ePcePGUVJSUu+5AwYM4NxzzwXgmmuuYdWqVQAsX76cs846i4yMDF555RXeeuutes/dvHkzaWlpjBo1CoCePXvWTGtMnDiRXr16kZKSwuDBg3n//ffj1v71r3+93vga+lwgFrC1g/Kiiy4iOTmZjIwMqqqqmDRpEsABn3MiY6nr0UcfJTU1lVtvvbVF+ygpKWH8+PH07duX7t27M2PGDFasWAHAYYcdxuTJk+uNuzW6/By4HFoOP/zwmuVu3brV3O/WrRv79+8HiPtf6erTt+KdxuXu5OTk8PDDDydcx9lnn81TTz3FaaedxnnnnceTTz7J6tWr+cEPftCsMSQlJcWdQqm7XfWYEukuCvXHaWZ8/vnn3HLLLZSWljJgwADy8/PjfqvW3Rs83a1u7dXveUPb1d6msc/lyCOPjPv8bt26kZycXLNd9eec6FhqKyoq4le/+lVN4LZkH429/7XrbOy9aQ4dgcshZ+zYsTXTFcXFxfTp04eePXsesP53v/sdH38ca28/ceJEFi9ezK5dsW4RH330UYNHlrVfY+7cuYwdO5bhw4ezfPlyDj/8cHr16lVv2+Tk5Jp527YYW2FhIVVVVVRUVLBixQqysrLqbffBBx+wevVqIDZnO2bMmJpw6tOnD59++mnNvDrAUUcdVTPPffrpp7Njx46aI/s9e/a0SRg19Lm0RGNjief999/nlltuYdGiRTVH+om+H7WdddZZ/OEPf+DDDz+kqqqKZ599lnHjxrVoDInQEbi0m856lk5+fj7f+ta3GDp0KEcccQQFBbGuyLNnz+aqq65ixIgRjBs3jpNOOgmAwYMH8/3vf5/s7Gy+/PJLkpOT+elPf8rJJ5/c4Gucd955bNu2jbFjx5KUlMSAAQM4/fTT426bm5vL0KFDGTFiBA8++GCrxnbZZZexevVqhg0bhpnx6KOPcvzxx9fb7owzzqCgoICbbrqJQYMGcfPNN3PEEUdw4403kpGRQXp6es0UCcR+aTpz5kxSU1NZvXo1hYWF3H777ezdu5fU1FRefvnlVtUNDX8uLXH00Uc3OJZ4FixYQGVlJZdddhkAJ5xwAi+++GLC70e1tLQ0Hn74YSZMmIC7c/HFFzNlypQWj6MpCV3Qoa10xAUddBrhwbNp0ybOOOOMji5DJGjx/h61+IIOIiLSOSnARUQCpQAXEQmUAlxEJFAKcBGRQCnARUQCpfPApf3k1//SSuv2t7tt99eErtpOti3Ubh37+OOPc8QRRxzQuyQRxcXFHHbYYZxzzjkALd7PoUwBLtIBOrKdbFVVFUlJSW22v5kzZ7boecXFxfTo0aMmwFu6n0OZplCky1A72YbbyRYXFzNhwgSuvvpqMjIyKC4uZty4cVx++eWceuqp5OXlsXDhQrKyssjIyODdd98F4Le//S1nnXUWw4cP54ILLojbAjU/P5+5c+eyY8eOmpasmZmZJCUl8f7778fdx9atW3n88cf54Q9/SGZmJitXrqzZD8S6Go4ePZqhQ4dy2WWX1bQ1GD9+PLNmzSIrK4tTTz21puvjoUoBLl2K2sk23E52zZo1PPjgg/zpT38CYN26dTz22GNs2LCBp59+mj//+c+sWbOGG264gR//+McAjBkzhtdee40333yTK6+8kkcffbTBcZ9wwgk1LVlvvPFGvvGNb3DyySfH3Ud6ejozZ87kO9/5DmVlZfXel2uvvZZHHnmE9evXk5GRwQMPPFDz2P79+1mzZg0/+tGPDlh/KNIUinQp1e1kgUbbyVaHYt12skuWLAEabicLsHfvXo477rgGa2isnWx1G9WmxtDW7WR79uxJVlZWzYUKAEaNGkVaWhoAX/nKV8jOzgZiLVmXL18OwPbt27niiisoLy/niy++OOD5DXn11VeZP39+zT9izd3H7t27+eSTT2qaQOXk5DB9+vSax+O1oj1U6QhcupT2bCdbfXS5efNm8vPzG62jbjvZlStXsnr16poe3ImOIZGWrLXH1Fhvo4ZaskLD79Xtt9/ObbfdxoYNG/j5z3/eZDvV8vJyrr/+egoLC+nRo0eL9tGUeK1oD1UKcDnkqJ1s4nbv3s2JJ54I0GR3wH379nH55ZfzyCOPcOqppza5j4Zasvbq1YvevXvXHME//fTT7dqSNWSaQpH2c5BP+0vUodhOtvq6k82Vn5/P9OnTOfHEExk9ejR/+ctfGtz2j3/8IyUlJcyePZvZs2cD8OKLLza4j6997WtMmzaNpUuX1sy5VysoKGDmzJl89tlnDBw4kKeeeqpF9Xd1aifbCmoneyC1kxVpPbWTFRE5BCjARUQCldAcuJltBfYAVcB+dx9pZscAhUA6sBW43N0/bp8yJRSNXfBWRBrX3Cnt5hyBT3D3zFrzMHlAkbsPAoqi+3IIS0lJobKystl/CEUkFt6VlZWkpKQk/JzWnIUyBRgfLRcAxcCsVuxPAte/f3+2b99ORUVFR5ciEqSUlBT69++f8PaJBrgDvzczB37u7vOAfu5eDuDu5WYW96tpZpYL5AI1p2VJ15ScnJzQN/VEpG0kGuDnuvuOKKRfMrOETyqNwn4exE4jbEGNIiISR0Jz4O6+I7rdBfwayAJ2mlkaQHS7q72KFBGR+poMcDM70syOql4GsoGNwDIgJ9osB1jaXkWKiEh9iUyh9AN+HZ0a1h14xt3/28xKgEVmdj3wATC9kX2IiEgbazLA3f09YFic9ZXAxPYoSkREmqZvYoqIBEoBLiISKAW4iEigFOAiIoFSgIuIBEoBLiISKAW4iEigFOAiIoFSgIuIBEoBLiISKAW4iEigFOAiIoFSgIuIBEoBLiISKAW4iEigFOAiIoFSgIuIBEoBLiISKAW4iEigFOAiIoFSgIuIBEoBLiISKAW4iEigFOAiIoFSgIuIBEoBLiISKAW4iEigEg5wM0syszfN7Pno/jFm9pKZbYlue7dfmSIiUldzjsDvBDbVup8HFLn7IKAoui8iIgdJQgFuZv2BS4D5tVZPAQqi5QJgaptWJiIijUr0CPxHwL3Al7XW9XP3coDo9rh4TzSzXDMrNbPSioqK1tQqIiK1NBngZjYZ2OXua1vyAu4+z91HuvvIvn37tmQXIiISR/cEtjkXuNTMLgZSgJ5m9ktgp5mluXu5maUBu9qzUBEROVCTR+Du/l137+/u6cCVwCvufg2wDMiJNssBlrZblSIiUk9rzgOfA1xoZluAC6P7IiJykCQyhVLD3YuB4mi5EpjY9iWJiEgi9E1MEZFAKcBFRAKlABcRCZQCXEQkUApwEZFAKcBFRAKlABcRCZQCXEQkUApwEZFAKcBFRAKlABcRCZQCXEQkUApwEZFAKcBFRAKlABcRCZQCXEQkUApwEZFAKcBFRAKlABcRCVSzrokpB0rPe6Hd9r11ziXttm8R6Rp0BC4iEigFuIhIoBTgIiKBUoCLiARKAS4iEigFuIhIoJoMcDNLMbM1ZrbOzN4yswei9ceY2UtmtiW67d3+5YqISLVEjsD/AZzv7sOATGCSmY0G8oAidx8EFEX3RUTkIGkywD3m0+hucvTjwBSgIFpfAExtjwJFRCS+hObAzSzJzMqAXcBL7v460M/dywGi2+ParUoREaknoQB39yp3zwT6A1lmNiTRFzCzXDMrNbPSioqKFpYpIiJ1NessFHf/BCgGJgE7zSwNILrd1cBz5rn7SHcf2bdv39ZVKyIiNRI5C6WvmR0dLacCFwBvA8uAnGizHGBpO9UoIiJxJNKNMA0oMLMkYoG/yN2fN7PVwCIzux74AJjejnWKiEgdTQa4u68HhsdZXwlMbI+iRESkafompohIoBTgIiKB0hV5RDpafq8Oet3dHfO60mZ0BC4iEigFuIhIoBTgIiKBUoCLiARKAS4iEigFuIhIoBTgIiKBUoCLiARKAS4iEigFuIhIoBTgIiKBUoCLiARKAS4iEigFuIhIoBTgIiKBUoCLiARKAS4iEigFuIhIoBTgIiKBUoCLiARKAS4iEigFuIhIoBTgIiKBUoCLiASqyQA3swFmttzMNpnZW2Z2Z7T+GDN7ycy2RLe9279cERGplsgR+H7gX939DGA0cKuZDQbygCJ3HwQURfdFROQgaTLA3b3c3d+IlvcAm4ATgSlAQbRZATC1nWoUEZE4mjUHbmbpwHDgdaCfu5dDLOSB4xp4Tq6ZlZpZaUVFRSvLFRGRagkHuJn1AJ4D/sXd/5bo89x9nruPdPeRffv2bUmNIiISR0IBbmbJxMJ7obsviVbvNLO06PE0YFf7lCgiIvEkchaKAb8ANrn7f9Z6aBmQEy3nAEvbvjwREWlI9wS2ORf4JrDBzMqidfcBc4BFZnY98AEwvV0qFDkY8nt1dAUizdZkgLv7KsAaeHhi25YjIiKJ0jcxRUQCpQAXEQmUAlxEJFAKcBGRQCnARUQCpQAXEQmUAlxEJFAKcBGRQCnARUQCpQAXEQmUAlxEJFAKcBGRQCnARUQCpQAXEQmUAlxEJFAKcBGRQCVyRR4R6Yo68ipE+bs77rW7EB2Bi4gESgEuIhIoBbiISKAU4CIigVKAi4gESgEuIhIoBbiISKAU4CIigVKAi4gESgEuIhKoJgPczJ40s11mtrHWumPM7CUz2xLd9m7fMkVEpK5EjsAXAJPqrMsDitx9EFAU3RcRkYOoyQB39xXAR3VWTwEKouUCYGrbliUiIk1paTfCfu5eDuDu5WZ2XEMbmlkukAtw0kkntfDl5JDRkR3yRALT7r/EdPd57j7S3Uf27du3vV9OROSQ0dIA32lmaQDR7a62K0lERBLR0gBfBuREyznA0rYpR0REEpXIaYTPAquB08xsu5ldD8wBLjSzLcCF0X0RETmImvwlprtf1cBDE9u4FhERaQZ9E1NEJFC6qLGIHHwddbpoF7uYso7ARUQCpQAXEQmUAlxEJFCaA+/M9LVyEWmEjsBFRAKlABcRCZSmUDqp9LwX2JrS0VWIdDEdOS3ZDqcw6ghcRCRQCnARkUApwEVEAqUAFxEJlAJcRCRQCnARkUApwEVEAqUAFxEJlAJcRCRQCnARkUApwEVEAqUAFxEJlAJcRCRQCnARkUApwEVEAqUAFxEJlAJcRCRQrboij5lNAh4DkoD57j6nTaqKp4VX0miLq9qkf/5M63ciItLGWnwEbmZJwE+Bi4DBwFVmNritChMRkca1ZgolC3jH3d9z9y+A/wKmtE1ZIiLSlNYE+InAtlr3t0frRETkIGjNHLjFWef1NjLLBXKju5+a2eZWvObB0Af48MBVkzukkHhvcAvEGU+wutJYQOPp7Np2PA+06m/0yfFWtibAtwMDat3vD+you5G7zwPmteJ1DiozK3X3kR1dR1vpSuPpSmMBjaezC2E8rZlCKQEGmdkpZnYYcCWwrG3KEhGRprT4CNzd95vZbcD/EDuN8El3f6vNKhMRkUa16jxwd38ReLGNauksgpnuSVBXGk9XGgtoPJ1dpx+Pudf7vaOIiARAX6UXEQmUAhwws383s/VmVmZmvzezE2o99l0ze8fMNpvZ/+nIOhNlZv9hZm9HY/q1mR1d67EQxzPdzN4ysy/NbGSdx4IbD8TaUEQ1v2NmeR1dT3OZ2ZNmtsvMNtZad4yZvWRmW6Lb3h1ZY3OY2QAzW25mm6I/a3dG6zv3mNz9kP8BetZavgN4PFoeDKwDDgdOAd4Fkjq63gTGkw10j5YfAR4JfDxnAKcBxcDIWutDHU9SVOtA4LBoDIM7uq5mjmEsMALYWGvdo0BetJxX/ecuhB8gDRgRLR8F/Dn689Wpx6QjcMDd/1br7pH88wtJU4D/cvd/uPtfgHeItRDo1Nz99+6+P7r7GrFz9CHc8Wxy93hfAAtyPHSBNhTuvgL4qM7qKUBBtFwATD2YNbWGu5e7+xvR8h5gE7FvlnfqMSnAI2b2oJltA2YA/xat7grtAr4N/C5a7grjqS3U8YRad1P6uXs5xAIROK6D62kRM0sHhgOv08nH1KrTCENiZi8Dx8d56H53X+ru9wP3m9l3gduA2STYLqAjNDWeaJv7gf3Awuqnxdk+mPHEe1qcdZ1iPE0Ite4uz8x6AM8B/+LufzNro4YW7eSQCXB3vyDBTZ8BXiAW4Am1C+gITY3HzHKINXGZ6NEEHgGPpwGddjxNCLXupuw0szR3LzezNGBXRxfUHGaWTCy8F7r7kmh1px6TplAAMxtU6+6lwNvR8jLgSjM73MxOAQYBaw52fc0VXWhjFnCpu39W66Egx9OIUMfTVdtQLANyouUcoKH/OXU6FjvU/gWwyd3/s9ZDnXtMHf1b1M7wQ+xf3Y3AeuC3wIm1Hruf2BkDm4GLOrrWBMfzDrE51rLo5/HAx3MZsaPWfwA7gf8JeTxR3RcTO9PhXWLTRB1eUzPrfxYoB/ZFn831wLFAEbAluj2mo+tsxnjGEJvGWl/r783FnX1M+iamiEigNIUiIhIoBbiISKAU4CIigVKAi4gESgEuIhIoBbh0KWY23szOqXV/ppld28J9XVenM+V8MxvcFnWKtAWdRihdipnlA5+6+9w22FcxcLe7l7Z2XyLtQUfg0umZ2W/MbG3Upzm31vpJZvaGma0zs6KoCdFM4DtRb/fzzCzfzO42szPMbE2t56ab2fpo+d/MrMTMNprZPIuZBowEFkb7SjWz4up+5GZ2lZltiJ7zSK39fho1RltnZq+ZWb+D9DbJIUgBLiH4trt/lVig3mFmx5pZX+AJ4BvuPgyY7u5bgceBH7p7pruvrN6Bu28CDjOzgdGqK4BF0fJP3H2Uuw8BUoHJ7r4YKAVmRPvaW72vaFrlEeB8IBMYZWZTo4ePBF6LaloB3NjWb4ZINQW4hOAOM1tHrLf5AGI9T0YDKzzWBxx3r9ubOp5FwOXR8hVAYbQ8wcxeN7MNxEL5zCb2MwoodvcKj/VdX0jsAgcAXwDPR8trgfQE6hJpEQW4dGpmNh64ADg7Oqp9E0gh1pK1ub/AKQQuN7NTAXf3LWaWAvwMmObuGcSO6lOaKquRx/b5P3+xVMUh1PFTDj4FuHR2vYCP3f0zMzud2JE3wGpgXNSFEDM7Jlq/h9glsepx93eJher/5Z9H39Vh/WHUC3parac0tK/Xo9fuY2ZJwFXAH1oyOJHW0NGBdHb/DcyMfuG4mdg0Cu5eEf1Cc4mZdSPWp/lCYt0kF5vZFOD2OPsrBP6D2DU0cfdPzOwJYAOwlVir12oLgMfNbC9wdvVKj/WG/i6wnNjR+Ive8EUnRNqNTiMUEQmUplBERAKlABcRCZQCXEQkUApwEZFAKcBFRAKlABcRCZQCXEQkUApwEZFA/X9gIiEOshohewAAAABJRU5ErkJggg==\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"model.eval()\n",
"model_norm.eval()\n",
"out=model.activation(validation_dataset[0][0].reshape(-1,28*28))\n",
"plt.hist(out[2],label='model with no batch normalization' )\n",
"out_norm=model_norm.activation(validation_dataset[0][0].reshape(-1,28*28))\n",
"plt.hist(out_norm[2],label='model with normalization')\n",
"plt.xlabel(\"activation \")\n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!--Empty Space for separating topics-->\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We see the activations with Batch Normalization are zero centred and have a smaller variance.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compare the training loss for each iteration\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEGCAYAAABvtY4XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAA0d0lEQVR4nO3deXxU9b3w8c93luwLIQlrlACiLAkECIiggMW6ValY1FpaFWu5trXeYqsX9eqj3udpfWm1on2spVbk8arlVgWX4oaKiAtbQWUVUJCwhi2E7DPze/74zQwTSEISMplw5vt+kVdmzpzlOzPke37ne37nd8QYg1JKKedxxToApZRS0aEJXimlHEoTvFJKOZQmeKWUcihN8Eop5VCeWAcQKScnx+Tn58c6DKWUOmWsXLlynzEmt6HXOlSCz8/PZ8WKFbEOQymlThkisq2x17REo5RSDqUJXimlHEoTvFJKOVSHqsErdSqrq6ujpKSE6urqWIeiHCgpKYm8vDy8Xm+zl9EEr1QbKSkpIT09nfz8fEQk1uEoBzHGsH//fkpKSujdu3ezl9MSjVJtpLq6muzsbE3uqs2JCNnZ2S0+OtQEr1Qb0uSuoqU1/7ccleA/2lTKtv0VsQ5DKaU6BEcl+OlzP+dvS76JdRhKxYSI8Jvf/Cb8/A9/+AP33Xdfs5d/9tlnyc3NpaioiEGDBjF58mQqKyubXGbRokV88sknTc6zdetWCgoKTrj9++67j5SUFPbu3RuelpaW1rzg28iiRYu47LLLAHjttdd48MEHW7yO1atXs2DBgvDz1q6nLUQ1wYtIJxF5SUQ2iMh6ETknmtur8fmprvNHcxNKdViJiYm88sor7Nu3r9XruOaaa1i9ejVr164lISGBuXPnNjl/cxJ8S+Tk5PDII4+0alljDIFAoM1imThxIjNmzGjxcscm+Naupy1EuwU/E3jLGNMfGAKsj+bG/AGDz693qFLxyePxMG3aNP74xz8e99q2bduYMGECgwcPZsKECXz77bdNrsvn81FRUUFWVhYAr7/+OmeffTZDhw7lggsuYM+ePWzdupWnnnqKP/7xjxQVFfHRRx+xZ88eJk2axJAhQxgyZEg4+fv9fn72s58xaNAgLrzwQqqqqhrc7o033sjcuXM5cODAca89+uijFBQUUFBQwGOPPQbYo4MBAwbwi1/8gmHDhvHRRx/Rv39/brrpJgoKCpgyZQoLFy5kzJgx9OvXj2XLlgGwbNkyRo8ezdChQxk9ejQbN248bnvPPvsst9xyCwBFRUXhn+TkZD788MMG11FbW8u9997L3LlzKSoqYu7cufXW09j3cMMNN3DrrbcyevRo+vTpw0svvdTk99NcUesmKSIZwFjgBgBjTC1QG63tAfgChrqAJngVe/e/vpZ1Ow+36ToH9sjgf10+qMl5fvnLXzJ48GDuuOOOetNvueUWrrvuOq6//nqeeeYZbr31VubPn3/c8nPnzmXJkiXs2rWLM888k8svvxyAc889l88++wwR4emnn+ahhx7ikUce4eabbyYtLY3f/va3gD0CGDduHPPmzcPv93PkyBEOHjzIpk2bePHFF/nrX//K1Vdfzcsvv8yPf/zj47aflpbGjTfeyMyZM7n//vvD01euXMns2bNZunQpxhjOPvtsxo0bR1ZWFhs3bmT27Nk8+eSTbN26lc2bN/OPf/yDWbNmMWLECF544QWWLFnCa6+9xu9+9zvmz59P//79Wbx4MR6Ph4ULF3LXXXfx8ssvN/q5rl69GrA7uoceeojRo0dTVVXV4DoeeOABVqxYwZ/+9CfA7iia8z3s2rWLJUuWsGHDBiZOnMjkyZOb/K6bI5ot+D5AKTBbRFaJyNMiknrsTCIyTURWiMiK0tLSk9qgzx+gztd2h2hKnWoyMjK47rrrePzxx+tN//TTT/nRj34EwE9+8hOWLFnS4PKhEs3u3bspLCzk4YcfBmwf/4suuig8be3atQ0u//777/Pzn/8cALfbTWZmJgC9e/emqKgIgOHDh7N169ZG38Ott97KnDlzOHz46A5yyZIlTJo0idTUVNLS0rjyyiv56KOPAOjVqxejRo0Kz9u7d28KCwtxuVwMGjSICRMmICIUFhaGt1tWVsZVV11FQUEB06dPb/T9RNq0aRO33347c+fOxev1tmodTX0PV1xxBS6Xi4EDB7Jnz54Trqs5onmhkwcYBvzKGLNURGYCM4B7ImcyxswCZgEUFxe3uvkdCBgCBnxtWINTqrVO1NKOpl//+tcMGzaMqVOnNjrPibrciQiXX345TzzxBDNmzOBXv/oVt912GxMnTmTRokUtOnkL9vxAiNvtbrREA9CpUyd+9KMf8eSTT4anGdN4akhNrd9ujNyWy+UKP3e5XPh8PgDuuecezj//fObNm8fWrVsZP358k/FXVFRw9dVX89e//pUePXq0ah0NifweIuNu6v22RDRb8CVAiTFmafD5S9iEHxX+4AdSpzV4Fec6d+7M1Vdfzd/+9rfwtNGjR/P3v/8dgOeff55zzz33hOtZsmQJffv2BWyLt2fPngDMmTMnPE96ejrl5eXh5xMmTODPf/4zYOvuka3wlrjtttv4y1/+Ek7IY8eOZf78+VRWVlJRUcG8efM477zzWrVuqP9+IksojZk6dSpTp06tt83G1nHsZxKpNd/DyYhagjfG7Aa2i8hZwUkTgHXR2p4/EErw2oJX6je/+U293jSPP/44s2fPZvDgwTz33HPMnDmzweVCJwcHDx7MqlWruOcee8B93333cdVVV3HeeeeRk5MTnv/yyy9n3rx54ZOsM2fO5IMPPqCwsJDhw4c3q2zRkJycHCZNmkRNTQ0Aw4YN44YbbmDkyJGcffbZ3HTTTQwdOrRV6wa44447uPPOOxkzZgx+f9M977Zt28ZLL73EM888Ez7RumLFikbXcf7557Nu3brwSdZIzf0e2oq01aFAgysXKQKeBhKAr4GpxpiDjc1fXFxsWnvDjyM1Pgr+19uMzO/M/9wc1d6YSjVo/fr1DBgwINZhKAdr6P+YiKw0xhQ3NH9UBxszxqwGGtxwW/MFW+612oJXSinAQVey+oIlGj3JqpRSlmMSfKgGrxc6KaWU5ZgEH2rBa4lGKaUsxyR4v19b8EopFckxCb4uWHv3aQteKaUAByV4f7hEoy14FZ/cbjdFRUUMGTKEYcOGnXCUx0OHDtW7WrQx48eP50Tdl7du3YqI8MQTT4Sn3XLLLc26iKgt5efnh/v/jx49ulXr+N3vflfveWvX0xE4JsGHSjPai0bFq+TkZFavXs3nn3/O73//e+68884m529ugm+uLl26MHPmTGprWzemYOiq1bbS2mGMj03wbTkccntzTILXXjRKHXX48OHwUL9HjhxhwoQJDBs2jMLCQl599VUAZsyYwZYtWygqKuL2228H4KGHHqKwsJAhQ4bUG8P8H//4ByNHjuTMM88MD/J1rNzcXCZMmFBvKIOQ1atXM2rUKAYPHsykSZM4eNBe7zh+/Hjuuusuxo0bx8yZMxk/fjzTp09n7NixDBgwgOXLl3PllVfSr18//vM//zO8viuuuILhw4czaNAgZs2a1WA8oZuF3HvvveErUHv27Bkeo6ehdcyYMYOqqiqKioqYMmVKvfUYY7j99tspKCigsLAwfJXqokWLGD9+PJMnT6Z///5MmTKlzcaSOVlRvdCpPYVa7tqLRnUIb86A3V+27Tq7FcIljd8ZKJSYqqur2bVrF++//z4ASUlJzJs3j4yMDPbt28eoUaOYOHEiDz74IGvWrAkPhfvmm28yf/58li5dSkpKSr0x2X0+H8uWLWPBggXcf//9LFy4sMEYZsyYwSWXXMKNN95Yb/p1113HE088wbhx47j33nu5//77w2O6Hzp0iA8//BCww/EmJCSwePFiZs6cyfe//31WrlxJ586d6du3L9OnTyc7O5tnnnmGzp07U1VVxYgRI/jBD35AdnZ2gzE98MADPPDAA5SVlXHeeeeFx2ZvaB0PPvggf/rTn8KfSaRXXnklfIS0b98+RowYwdixYwFYtWoVa9eupUePHowZM4aPP/446uPMNIdjWvDhC500was4FSrRbNiwgbfeeovrrrsOYwzGGO666y4GDx7MBRdcwI4dOxocjnbhwoVMnTqVlJQUwA5aFnLllVcCJx7qt3fv3owcOZIXXnghPK2srIxDhw4xbtw4AK6//noWL14cfv2aa66pt46JEycCUFhYyKBBg+jevTuJiYn06dOH7du3A3ZMlyFDhjBq1Ci2b9/Opk2bmvxsjDFMmTKF6dOnM3z48FatY8mSJVx77bW43W66du3KuHHjWL58OQAjR44kLy8Pl8tFUVFRk59Re3JOCz5YmgkYW65xu/Tu9iqGmmhpt4dzzjmHffv2UVpayoIFCygtLWXlypV4vV7y8/Oprq4+bhljTKPDCIeGsnW73Sesld91111Mnjw53Lo9kcaG+40c6jf03OfzsWjRIhYuXMinn35KSkoK48ePb/D9RLrvvvvIy8sLl2das46myi7HDofc1ucTWssxLXh/xJ2cdERJFe82bNiA3+8nOzubsrIyunTpgtfr5YMPPmDbtm3A8cPaXnjhhTzzzDPhG203dNu85ujfvz8DBw7kjTfeACAzM5OsrKxw7f65554Lt+Zbo6ysjKysLFJSUtiwYQOfffZZk/O/8cYbvPvuu/VugtLUOrxeL3V1dcetZ+zYscydOxe/309paSmLFy9m5MiRrX4f7cE5LfiI3jM+vW2fikOhGjzY1uacOXNwu91MmTKFyy+/nOLiYoqKiujfvz8A2dnZjBkzhoKCAi655BIefvhhVq9eTXFxMQkJCVx66aXH9ShprrvvvrvecL5z5szh5ptvprKykj59+jB79uxWv8+LL76Yp556isGDB3PWWWfVu5tTQx555BF27twZTsYTJ07k7rvvbnQd06ZNY/DgwQwbNoznn38+PH3SpEl8+umnDBkyBBHhoYceolu3bmzYsKHV7yXaojpccEudzHDB763fw0/n2GVX3fNdslIT2jI0pU5IhwtW0dbS4YIdU6KJbLXXaV94pZRyUIKP6P+ufeGVUspJCT6i1a4nWVWsdKSSp3KW1vzfckyCr9+LRv/IVPtLSkpi//79muRVmzPGsH//fpKSklq0nIN60USUaLQGr2IgLy+PkpISSktLYx2KcqCkpCTy8vJatIxjEny9FrxPW1Cq/Xm9Xnr37h3rMJQKc0yJJnKIAu1Fo5RSTkrwAe1Fo5RSkRyT4HWoAqWUqs8xCd6nCV4ppeqJ6klWEdkKlAN+wNfY5bRtQbtJKqVUfe3Ri+Z8Y8y+aG8kstWuY8IrpZSDSjT1WvA6mqRSSkU9wRvgHRFZKSLTGppBRKaJyAoRWXEyF4jUq8H7tAWvlFLRTvBjjDHDgEuAX4rIcbd4McbMMsYUG2OKc3NzW70hv17JqpRS9UQ1wRtjdgZ/7wXmAVG7/Ulk33c9yaqUUlFM8CKSKiLpocfAhcCaaG1PR5NUSqn6otmLpiswL3gTXw/wgjHmrWhtzBcwJHldVNcF9EpWpZQiigneGPM1MCRa6z+W329I8rqprgvoWDRKKYWDukn6AoZkrxvQ0SSVUgoclOD9gQAJHhci2otGKaXAQQm+LmDwuASvy0WtnmRVSinnJHi/3+BxufC6RU+yKqUUDkrwvoDB7RI8bpeORaOUUjgowfsDATxuwesWarUFr5RSzknwoRa8V1vwSikFOCnB+w1elwuPW+oNPKaUUvHKMQneH9GC1140SinloATvC9XgXVqiUUopcFCC94d70Wg3SaWUAgcleF/oQict0SilFOCkBK8XOimlVD3OSfCBAG634HG5dCwapZTCQQneHyrReFx6oZNSSuGgBB++0Mkl2otGKaVwUIIPteA9btFb9imlFA5K8HV+g8ftCg5VoCUapZRyTIL3BwLhbpJ6yz6llHJQgj862JjoLfuUUgoHJfijNXjtJqmUUuCgBG9b8C68LqFOa/BKKYUn1gG0FZ8/YK9idbu0F41SSuGQFnwgYAgYIm7Zpy14pZSKeoIXEbeIrBKRN6K1Db+xCd0TOskaCGCMJnmlVHxrjxb8vwPro7kBf/AOTm6X7QdvzNFpSikVr6Ka4EUkD/ge8HQ0txO6RV/oStbIaUopFa+i3YJ/DLgDaPSsp4hME5EVIrKitLS0VRsJjT0TuqMToGPCK6XiXtQSvIhcBuw1xqxsaj5jzCxjTLExpjg3N7dV24pswXtDLXg90aqUinPR7CY5BpgoIpcCSUCGiPy3MebHbb2hyBq8iH2sI0oqpeJd1Frwxpg7jTF5xph84IfA+9FI7tBwC15LNEqpeOeIC538/lALXhDREo1SSkE7JXhjzCJgUbTWHxo90uOOSPA6Ho1SKs45owUfLtG4cAeLTrU6oqRSKs45IsH7Iko0Hpe24JVSChwyFo2/gQuddERJpVS8c0SCD7XW3W4hIVij0REllVLxziEJPrIFb9+S9qJRSsU7ZyR4/9GTrEdLNNqCV0rFN0ck+HANXks0SikV5ogEH67B62iSSikV5ogEX68XjUtb8EopBQ5J8HUR/eCPlmi0Ba+Uim+OSPChFrzXffQkq44mqZSKd45I8JE1eK+eZFVKKcAhCd7fwHDBWqJRSsU7RyR4XyBiLJrQhU46Fo1SKs45I8FHXOikLXillLKaleBF5LnmTIsVf+D4m25rDV4pFe+a24IfFPlERNzA8LYPp3Uix6JxuQSX6Fg0SinVZIIXkTtFpBwYLCKHgz/lwF7g1XaJsBn8ETV4sN0ltQWvlIp3TSZ4Y8zvjTHpwMPGmIzgT7oxJtsYc2c7xXhCvog7OkEowWsLXikV35pbonlDRFIBROTHIvKoiPSKYlwtErqoKdSC97hFW/BKqbjX3AT/Z6BSRIYAdwDbgP8XtahaKLIGD7YFr90klVLxrrkJ3meMMcD3gZnGmJlAevTCahl/wOAScAUTfILbRY1PE7xSKr4196bb5SJyJ/AT4LxgLxpv9MJqGV/AhOvvAN0yk9h+oDKGESmlVOw1twV/DVAD3GiM2Q30BB6OWlQt5A+YcP0dYGD3DNbvKiegY8IrpeJYsxJ8MKk/D2SKyGVAtTGmyRq8iCSJyDIR+VxE1orI/W0Qb4Pq/IFw/R1gYI8MjtT4KDlYFa1NKqVUh9fcK1mvBpYBVwFXA0tFZPIJFqsBvmOMGQIUAReLyKiTiLVR/oAJDxMMMKB7BgDrdpVFY3NKKXVKaG4N/m5ghDFmL4CI5AILgZcaWyB4UvZI8Kk3+BOVmokvYHBH1ODP6pqOS2DdrnIuLugejU0qpVSH19wavCuU3IP2N2dZEXGLyGrsla/vGmOWNjDPNBFZISIrSktLmxlOfX6/qVeiSU5w0yc3jXU7D7dqfUop5QTNTfBvicjbInKDiNwA/BNYcKKFjDF+Y0wRkAeMFJGCBuaZZYwpNsYU5+bmtiD0o3zHnGSF0IlWTfBKqfh1orFozhCRMcaY24G/AIOBIcCnwKzmbsQYcwhYBFzc6kib4AsE6tXgwdbhdxyqoqyyLhqbVEqpDu9ELfjHgHIAY8wrxpjbjDHTsa33x5paUERyRaRT8HEycAGw4STjbZDtB39MC75H6ESrtuKVUvHpRAk+3xjzxbETjTErgPwTLNsd+EBEvgCWY2vwb7QqyhOwNXgXvHQjrH4RsCUa0ASvlIpfJ+pFk9TEa8lNLRjcMQxtcUSt4AsYMqmANS+DJwmKriU3PZHc9ETW7tSukkqp+HSiFvxyEfnZsRNF5KfAyuiE1HL+QICzzGb7pOZoi724VxafbtmP7bGplFLx5UQt+F8D80RkCkcTejGQAEyKYlwt4gsY+vm32Cc15eHpY8/M5c01u9m89wj9unaYsdGUUqpdNJngjTF7gNEicj4Q6uL4T2PM+1GPrAV8fkM/f6gFXz/BA3z4VakmeKVU3GnWlazGmA+AD6IcS6v5A4Y+daEEfyQ8vWenZPrmprJ40z5uOq9PjKJTSqnYaO6FTh1aoq+Mrv5d9klECx5sK37p1/uprvPHIDKllIodRyT43rWb7IPcAccn+H651PgCLPvmQAwiU0qp2HFEgs8PlWfyz4Xacoi4Xd/ZfTqT4Hax+KvWjXOjlFKnKkck+DN8m9jn6QadTrcT6irCr6UkeBjVN5s31+zWG4AopeKKQxL8FrYnnQmJwZ4yx5Rprhzakx2HqliqZRqlVBw59RO8r5Z9ksU3KYWNJviLBnUjLdHDy/8qiUGASikVG6d+gvck8IvE37Ek52pItOPPHJvgkxPcfK+wOwu+3EVFjS8GQSqlVPs79RM89kInj0siWvDHDzD2g+F5VNb6eWvN7naOTimlYsMZCT50y77ENDsh4mKnkBH5WfTKTuHPH27hQEVtO0eolFLtzxEJ3h8I4HVLozV4ABHh/1xRyPYDlVw76zNKy2vaOUqllGpfzb3pdocWvmVfIzX4kHP75TD7hhH8dM4Kxj70AcX5WYzqk83ovtnkZ6ey70gNHreL/OwUROwNRIwx4cdKKXUqcUSC94fu6JQQKtE0nOABRp+Rwz9uPoeXVpbwyZZ9PPz2xuPm6dkpmb5d0vhqdzlHanyc378LFwzowqAemeRnp+BxN//AxxhDjS9AdZ2fqjo/tb4AOWmJpCY64qNXSnVgjsgy08b2oei0TuBJAHeivZq1CQU9MynomQlAaXkNn329nz2Hq8lNT6S82seHX5VScrCKc/pm43ULC9fv5fXPdwIgAmmJHjKSvKQneUhJcFNZaxN3epKHtCQP+4/Usu9IDRU1fqp9fhoajj4nLYGMJC8JHheJHlfwt5sEj4sEt31+pMbHrrJqvG6hZ6dkUhM91PgC1Prs9mr9AWrq7O9aX4CAMXjdLkSg1mev5u2cmkBmsheXCCK2VCWAK3hUUusL4AsE8LpdeNwuaur81Pjs8ySvi2SvjamixkdZVR2+Bi4WC70/g6Gq1s/hah9uEdKSPKQl2s/E6xL8BgIBgz9gMNijLpdI+LdLBIPdIYbiCL2Po/GDYB+HVNT4qK4L0CnFS1ZKAv6AwRcIkOhxk+R1Y4yh1h+gzh+gzm/CR2US/D4Jrs8YQ8DYBkPAGBI9bjKTvRgMZZV1HKqq41BlLf6AISXBvrfURDeJHnc4ntBvuw77fgPG4A9+RgluQUSorvNTUxcgOcFNstdNRa2Pylo/bpfY/w9uV/i7jPyMQ5/zsZ99SEqCjbnWb9h/pIYaX6De65Gfm9SbXv8otf5rjb3S+PqOf03C0zqleMlNSyQl0YPX7cLrFrxuF6dlpVDYM5PMFC+qbUhHuhlGcXGxWbFixcmt5KG+MHAiXPbHtgkK8PkDrN9Vzld7ytm2v4LD1T4OV9dRXu2jqtZPktdNotdFebWP8uo6slMTyE1PIj3JQ5LHRaLX/hEned143cLe8hq2H6ikvMZnE3Xoxx+gJpS8fQFSEz10y0iiLmDYcbCSqlo/iV53eAcQ2jGEdgouEXyBAP6AIcnrJmAMByvqKKuqI2BsWggYA/YfxhgSPC48Lhe+gE1+icH11voNNXV+quv8VPsCpCbaxOGNOHppKEEke92kJ3kwBo7U+CivsZ+JP2BwB5O02yUIgt+YiARowiNMJHpdJHncJHldR4+WjE1sAWPjtvHbhJGa4CHR46Ksqo4DlbV4XILH5aLG56eq1o87mEAS3C48brttg8EYu47QZ3F0ZwMul1BdF6CsshYRCe88MlO8uEWoqPFRUeujosZPjc8OZHd0RwduEVzBdbldEv58fP5AcAdhd5xVtfbILiXB7ix8/qM7o9pjk3PEJ95Y1bCy1s+hyloSPC46pyaQnBDRhov4W4/8qz82BTS2Azl+vsjXGs8jkS8FjOFgZR37K2oabPiA3Ukle930yU1l3Jm5nNk1Ha/bRb+uaeRlpTS6nXglIiuNMcUNveaIFnw9ielNlmhaw+N2UZiXSWFeZpuuV6l45Q+YcKMmtDP7urSCL3eUsf9IDRW1ftbsKOMP73wVXsYlcGlhd64uPo3+3dLJTU/U82MnoAleKdXu3C6x5Snc4Wk9OiVzbr+cevPtO1LD7rJqanwB3lm3mxc++5Y3vrBDg+ekJVDcqzPn989l0tA8EjyO6BTYphyY4DM0wSvlEDlpieSkJQIwvFcWt5x/Bl+UlPHVnnK+3FHG8q0HeGvtbh5/bzNTx+Qz5LRODOieQZp2YgAcmeDT4PDOWEehlIqC9CQvY87IYcwZtqVvjGHxpn08+s5G/vc/1wOQ4HFx0aBuXDviNM7pmx3XZRwHJngt0SgVL0SEcWfmMrZfDrvKqtm4u5xFG/cyb9UOXv98J2d2TeOG0b2ZNLQnyQnuE6/QYTTBK6VOeSJCj07J9OiUzPn9u3DnpQN4/fOdzP54K3fN+5KH3t7AxCE9yEpJICvFy6SheXHRHTNqCV5ETgP+H9ANCACzjDEzo7W9ME3wSsW9JK+bq4pPY/LwPJZ9c4BnP9nK35dvD3c9feSdr5g6Jp8rhvakT25ajKONnmi24H3Ab4wx/xKRdGCliLxrjFkXxW3aBO+vAV+tvfBJKRW3RISz+2Rzdp9swF54tn73YZ54bzOPv29/zuiSxvXn9GLy8NMcV8aJWoI3xuwCdgUfl4vIeqAnEN0EnxAccKz2CHg6R3VTSqlTi8slDOqRyVM/Gc6OQ1UsXLeHeat2cM+ra3n47Y3kZaWQnZbApYXdmTS0J0neUzvht8uVrCKSDywGCowxh495bRowDeD0008fvm3btpPb2Krn4dVfwL9/Dln5J7cupZTjGWNYvvUgL68sYd+RGrYdqGTz3iNkJnsZ2bszBT0yKczLoKBHJl0ykmId7nFieiWriKQBLwO/Pja5AxhjZgGzwA5VcNIbbGLIYKWUOpaIMLJ3Z0b2tkf8xhiWfnOA/1m+ndUlh1i4fk94WIVe2Sm2m2bfHEb3zSYrtWOXgaOa4EXEi03uzxtjXonmtsI0wSulToKIMKpPNqOCdfsjNT7W7zrMFyVlfLplP6+t3skLS79FBHpkJtMrO4WctETSkjwU9MjkokFdyQ5enBVr0exFI8DfgPXGmEejtZ3jhBP88Xd1UkqplkpL9DAivzMj8jvz03N74/MH+GJHGZ9s3sfmvUfYur+SL0oOUVZVxwtLv+WeV9dweucUMpO95GenMKxXFoN6ZNI7J5WsFG/4wquKGjtoYffM5KjFHs0W/BjgJ8CXIrI6OO0uY8yCKG6zyfuyKqXUyfK4XQw7PYthp2fVm26MYf2uct5as4tv9ldysKKWT7bsZ/7qo1fW98lNZerofGr9hife38ShyjrO6prOBQO7MP2CM1t0r4lmxdqma4tgjFnC8UNER5+WaJRSMSAiDOyRwcAeGeFpxhh2HKriqz3lbNlbwRtf7OSeV9cCcF6/HM49I4cPNu7lvfV7uf2i/m0ekzOvZAVN8EqpmBMR8rJSyMtK4Tv94abzevN5SRk+f4DifHtS99/G9aXOHzjBmlrHeQnemwqI7QevlFIdiIjYu88dw9vGpZkQ5w2g7HLZe7NqC14pFeecl+DBlmmq9SSrUiq+OTPBp+XCkT2xjkIppWLKmQk+vQeU74p1FEopFVMOTfDdNMErpeKeMxN8Rg+o3A++mlhHopRSMePMBJ/e3f7WVrxSKo45M8FnBBP8YU3wSqn45cwEn97D/i7f2fR8SinlYM5M8NqCV0ophyb4pE7gSdYavFIqrjkzwYvYVvxhLdEopeKXMxM82J402oJXSsUxTfBKKeVQzk3wGd3tSVZz8vfxVkqpU5FzE3x6D/DXQNXBWEeilFIx4dwEH+4qqSdalVLxybkJPnyxk9bhlVLxybkJXlvwSqk459wEn9bN/i7dAO89AGtejm08SinVzpx30+0QTwKk5sJnT9rn3QZDwQ9iG5NSSrUj5yZ4gH4XwuEd9ibcm96FgB9c7lhHpZRS7SJqJRoReUZE9orImmht44SueBKuexXOutR2mTzwTcxCUUqp9hbNGvyzwMVRXH/zdRlgf+9dF9s4lFKqHUUtwRtjFgMHorX+Fsk9CxDYuz7WkSilVLuJeS8aEZkmIitEZEVpaWl0NpKQCln52oJXSsWVmCd4Y8wsY0yxMaY4Nzc3ehvqMlBb8EqpuBLzBN9uugyA/ZvBVxPrSJRSql3EV4I3fti3KdaRKKVUu4hmN8kXgU+Bs0SkRER+Gq1tNUvXQfa3lmmUUnEiahc6GWOujda6W6VzX3B5Yc8a2FcEiRmQ3jXWUSmlVNTET4nGkwA5/eDjx+BPxfD0BKitiHVUSikVNfGT4AHG/QeMnAbfuQfKtsNHj8Q6IqWUihpnj0VzrEFX2B+wPWo+fhyG/AhyzohlVEopFRXx1YKP9N0HwJsMr90CtZWxjkYppdpc/Cb4tC7wvUfh28/gv38A1WWxjkgppdpU/CZ4gMFXweRnoGQZ/HUCrHnFDimslFIOEF81+IYUXAnJneDN/4CXpkKnXjDgchh8NXQfEuvolFKq1eK7BR/S9zvwi89g8mzblXLZLNui36ODkymlTl2a4ENcbtua//HLMH0tJGXAq78Evy/WkSmlVKtogm9IWhe49GHY+a+j93RVSqlTjNbgGzPoSnvS9d17YdfnMGgSbF0C5bvgu/fb8eWVUqoD0wTfGBG44s+w5FFYOgvWvATuRHB7baK//DFbo/9mMXiTIL0bnHsbZPeNdeRKKQWAGGNiHUNYcXGxWbFiRazDOF7FftjzJeSNgMM74fmr4OA3gECPoYCxwxAbY1v3aV2gohSyetueOKk5sX4HSimHEpGVxpjihl7TFnxzpGZDn/H2cU4/+Nn7sPk96HUOZObZ6WU7YP7NsOC3xy8/4ib47n9BwAcb37QndDN6gLigrgoOfWt3GH2/A73HttvbUko5m7bg21IgANs/g4Q022rfvxnWv267XWaeDpX7oa6pESwFzrsNxt8Fbo89IvjwIfjmQ6irtDuA79xjy0dKKYW24NuPywW9Rh99ntHDtsjPugTe/9/QZywMu96ORV++087jSbLzpWTDW3faES63fQI/+JvdMXz8GPQYZuv/Hz1ihzi++MHjk3wgAP4aO76OUkqhLfiO54v/gdd/DRjbai++0Y6ZA/DOf8Knf4I+50Pn3jbp+6ptiadkBdSUQVo3yCuGC/8LOvexd7DasdLuaBLS7A7j60W2V9DwGyA5q+E4SlbA4j/YI4rTRrbLW1dKtVxTLXhN8B3Rvs0w/+eQexZc/rg9MgBbsnn/v2Dda1B9yN5A3JMIqV1sEs7oAQe32rKQvw5OP9sm8xB3IvhroWuBPWmM2COIxDToORxOO9sO23Dga/j0SXsP28RMuOF18CTDytl2m0mZtpto9hn2WoENC6Cm3B49DLoCRv0Cag7Dgjtg71rwptpYzvutvYDsWNuX29IW2B1U/+9BQkoUP2ClnEMTfLw5vBMW3G5b4cOvtwnz6w9t8h9xE3QdCLvX2B1BXQVUHoRvP4UDW46uY8i1MPpX8MI1UHXQloY8iZCQakfeDERc4dutEDJPs9cI7FwF/S6E/Vvg0DZ73qDmiF1/Wlfod4HdgZkAdCuw5ym+WVw//sRMW86qKQd3AgyfCmdebHd0fp/dqez+0pa1PIm2pFW6AS76ffPG9jfGzp/Rs+EdTlszpuXnTSoPwL6vbHnOkxCduJQjaIJXzVN1yPbqEdfR+9Xu3wLzbrbnFkb/yp48DgRs8t63ySbUzn3svMbA0r/YUlJyFlw95+g5iZKV8NYMe3SQe5adtnuNbamP/pXdobi9sOsLe6Swc5Vdx+FdcLgEUnPt0UbVQag9Uj9ul9fuCFKy4afvQEb3o6+V7bBlrdXPQ3Y/OOMC2PSO3Ukkd4axvz3ac2nfV3a7PYfDwCvsUdAnj9u4Rk6zRyilG+1n1H2ITdq1lXBkj41NxO4IkzKPfk7v3gOr/huG/QTO+K7dqW79yM6f1tXugPtdBOtfhdUv2uVrDsOetYCxO7arn2t5kg8EbM+sxAxIy23ZsuqUogleta/9WyCpk+1e2pTQ/72mWrd+H6ybD5sXAmLLSaefY88zVB+2yb5boU3Oz15mW+VnXWLPTWxfaq9CRmDAZfYIZtfndodU/FO7zq8/qL89cdmjizMvgbKSYCkLyMizO70dKyOed4Ndq+sfzQC4PDDkh3ZHsOZlu8PYudqWvDxJR3coe9fbW0cmpNn3kZVvj4Q8Sfb9iQs++D82lq6D7PvpVgiFk+2OYMe/oNPp9igpuZNN6lveg+VPw9aPobbcvvfTR9n1JWfZz/PwDjv/4GvstRo7VkD5Hvv+ElLt0UN1md3BBers+6urgvLddlr/70GvMfaIqvqwPZo68DV0GQjZfWDDP+Grt2wJL/9cSO9hLwasPGg7FxzeBRV7bUnwjAvs+3j3XvuddSuAXufCmRc1fHRVvsd+P5E7cQBfLVTus48T0+1PS/nrbElz+zL7/R170aKvxjYkmvr/euBrew4tvZvtDJHSueVxtJAmeBUfvl4EL99kk47LbS9C6z3WHh1k9bLzVOy3yc3lts9LVtiSFsYOFZ3bH5b/1fZ6SsyAiY/bBPz+f9nWetG1NlGuf8MeTZx+tj0y8NfaBO5NtUcBK5+1vZq+cw+c9xt7InznKns9RXInu22/z+4AvnoLBn4fBkw8er4lZOlf4M07QNy2tFa60W4rkrjtjs/vsyW3tK7Q/zLoUWTf24Y37NGWr9rOn5Jjz+EEfHZZ08x7ILgT7E7HV20/A19t491+03vYJH7szu9YnXrZzyatq73GZPeXNjZ3gv2sBky0Fw6WboBN79qryEXgrEvtTmvvenu0U7rR7ozAxpg3wu5wDm61iXnkTfaozOW2iXz3F7D/a+g5zG576VPw6f+FqgN2HQlpdjyqjJ42po1vwref2J31ubfZnf/aeXaHGNp5Vh2ET54IXt9SaY86z/43+9ubDIjd7rrXAAMj/81+p+tes+XNHz7fvO/hGJrglWqp8t32jzIps3XLH95lE9fpZ598LAe32iOi5E62df3V27YE1GOoPVra8r5NLuKyyWbAxIZLOqHymycRKvbZncuRPbYl3amXfVxbYdedlGlLUy6v/e1JtDHUVdkdxjeLj5Z/uhbYo49Qou0z3p70r62wRzxVB23CS86C9O5Hz32snWfLV90Gw/gZdlogACXLYf1r9ufQt0fjz+4HBT+wO7h/zbHXlWT0tEc3XQfZoxnEHqFsfs+eU+rc157L2b/JJnKXxy4X2tmB3Zn4a22pbPgNtoT46i/teaOQLgNtY2HDP+1RF0CXQfZoonQDEMyjvc6FSU/Zo4l5P4fS9fW/A3FDn3G2sRDqWOBJsuetJj9jP+sW0gSvlDr1GGNbz3WVNulGdun11RzdaZxIIGDLfBsXHN1R9RxuSzDbl9kEPeRau3MM8fvs/Inp0GWALbmAPWrZ9LbdIXYfbKfVVkBdtb04MTHjaAnHGLtzqTpgXwd7NBIq25SssC33PufbI7BWilmCF5GLgZmAG3jaGPNgU/NrgldKqZZpKsFHbTx4EXED/xe4BBgIXCsiA6O1PaWUUvVF84YfI4HNxpivjTG1wN+B70dxe0oppSJEM8H3BLZHPC8JTqtHRKaJyAoRWVFaWhrFcJRSKr5EM8E31Fn0uIK/MWaWMabYGFOcm6sXZCilVFuJZoIvAU6LeJ4H7Izi9pRSSkWIZoJfDvQTkd4ikgD8EHgtittTSikVIWrjwRtjfCJyC/A2tpvkM8aYtdHanlJKqfqiesMPY8wCYEE0t6GUUqphHepKVhEpBba1cvEcYF8bhhMNGuPJ6+jxgcbYVjTG5ulljGmwh0qHSvAnQ0RWNHY1V0ehMZ68jh4faIxtRWM8edE8yaqUUiqGNMErpZRDOSnBz4p1AM2gMZ68jh4faIxtRWM8SY6pwSullKrPSS14pZRSETTBK6WUQ53yCV5ELhaRjSKyWURmxDoeABE5TUQ+EJH1IrJWRP49OL2ziLwrIpuCv5txO5qox+oWkVUi8kZHjFFEOonISyKyIfh5ntORYhSR6cHveI2IvCgiSR0hPhF5RkT2isiaiGmNxiUidwb/hjaKyEUxiu/h4Pf8hYjME5FOsYqvsRgjXvutiBgRyYlljCdySif4DnxTER/wG2PMAGAU8MtgXDOA94wx/YD3gs9j7d+ByBtHdrQYZwJvGWP6A0OwsXaIGEWkJ3ArUGyMKcAOyfHDDhLfs8DFx0xrMK7g/80fAoOCyzwZ/Ntq7/jeBQqMMYOBr4A7YxhfYzEiIqcB3wW+jZgWqxibdEoneDroTUWMMbuMMf8KPi7HJqWe2NjmBGebA1wRkwCDRCQP+B7wdMTkDhOjiGQAY4G/ARhjao0xh+hAMWKH+0gWEQ+Qgh0xNebxGWMWAweOmdxYXN8H/m6MqTHGfANsxv5ttWt8xph3jDG+4NPPsCPQxiS+xmIM+iNwB/WHP49JjCdyqif4Zt1UJJZEJB8YCiwFuhpjdoHdCQBdYhgawGPY/6iBiGkdKcY+QCkwO1hGelpEUjtKjMaYHcAfsC25XUCZMeadjhJfAxqLqyP+Hd0IvBl83GHiE5GJwA5jzOfHvNRhYox0qif4Zt1UJFZEJA14Gfi1MeZwrOOJJCKXAXuNMStjHUsTPMAw4M/GmKFABbEvGYUFa9jfB3oDPYBUEflxbKNqlQ71dyQid2PLnM+HJjUwW7vHJyIpwN3AvQ293MC0mOeiUz3Bd9ibioiIF5vcnzfGvBKcvEdEugdf7w7sjVV8wBhgoohsxZa2viMi/03HirEEKDHGLA0+fwmb8DtKjBcA3xhjSo0xdcArwOgOFN+xGourw/wdicj1wGXAFHP0Ip2OEl9f7M788+DfTR7wLxHpRseJsZ5TPcF3yJuKiIhg68brjTGPRrz0GnB98PH1wKvtHVuIMeZOY0yeMSYf+7m9b4z5MR0rxt3AdhE5KzhpArCOjhPjt8AoEUkJfucTsOdbOkp8x2osrteAH4pIooj0BvoBy9o7OBG5GPgPYKIxpjLipQ4RnzHmS2NMF2NMfvDvpgQYFvx/2iFiPI4x5pT+AS7FnnHfAtwd63iCMZ2LPTz7Algd/LkUyMb2XtgU/N051rEG4x0PvBF83KFiBIqAFcHPcj6Q1ZFiBO4HNgBrgOeAxI4QH/Ai9rxAHTYR/bSpuLClhy3ARuCSGMW3GVvHDv3NPBWr+BqL8ZjXtwI5sYzxRD86VIFSSjnUqV6iUUop1QhN8Eop5VCa4JVSyqE0wSullENpgldKKYfSBK8cQUQ+Cf7OF5EftfG672poW0p1dNpNUjmKiIwHfmuMuawFy7iNMf4mXj9ijElrg/CUalfagleOICJHgg8fBM4TkdXBsdrdwXHGlwfHGf+34PzjxY7Z/wLwZXDafBFZGRzffVpw2oPY0SJXi8jzkdsS62GxY8F/KSLXRKx7kRwdx/754JWuiMiDIrIuGMsf2vMzUvHHE+sAlGpjM4howQcTdZkxZoSIJAIfi8g7wXlHYscf/yb4/EZjzAERSQaWi8jLxpgZInKLMaaogW1dib3SdgiQE1xmcfC1odixwXcCHwNjRGQdMAnob4wxkTe0UCoatAWvnO5C4DoRWY0dsjkbO04IwLKI5A5wq4h8jh2L/LSI+RpzLvCiMcZvjNkDfAiMiFh3iTEmgL3sPh84DFQDT4vIlUDl8atUqu1ogldOJ8CvjDFFwZ/exo7ZDnb4YTuTrd1fAJxjjBkCrAKSmrHuxtREPPYDHmNvZjESO8roFcBbLXgfSrWYJnjlNOVAesTzt4GfB4dvRkTODN405FiZwEFjTKWI9MfeajGkLrT8MRYD1wTr/LnYu081OoJg8P4AmcaYBcCvseUdpaJGa/DKab4AfMFSy7PYe7rmY8ftFuwdoq5oYLm3gJtF5AvsaICfRbw2C/hCRP5ljJkSMX0ecA7wOXb00DuMMbuDO4iGpAOvikgStvU/vVXvUKlm0m6SSinlUFqiUUoph9IEr5RSDqUJXimlHEoTvFJKOZQmeKWUcihN8Eop5VCa4JVSyqH+PyIU/cXshuuUAAAAAElFTkSuQmCC\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# Plot the diagram to show the loss\n",
"\n",
"plt.plot(training_results['training_loss'], label='No Batch Normalization')\n",
"plt.plot(training_results_Norm['training_loss'], label='Batch Normalization')\n",
"plt.ylabel('Cost')\n",
"plt.xlabel('iterations ') \n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Compare the validating accuracy for each iteration\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAX4AAAEGCAYAAABiq/5QAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8vihELAAAACXBIWXMAAAsTAAALEwEAmpwYAAAoPUlEQVR4nO3deXwV9b3/8dcnC1mAhECCbLIpikIgLCKyW7x91FZR/OFySyvFtl5bl7pUi1ir7e8+Wn+2VsHe1mqVYqstdUGt9fZeN5S0bqBYq2JBC0oJmCAEIYRsn98fZxKSkGUImZyE834+HnmcM3Nm+ZzJOe9MvjPzHXN3REQkcSTFuwAREelYCn4RkQSj4BcRSTAKfhGRBKPgFxFJMCnxLiCM3NxcHzp0aLzLEBHpUtauXVvi7nmNx3eJ4B86dChr1qyJdxkiIl2KmW1uaryaekREEoyCX0QkwSj4RUQSjIJfRCTBKPhFRBKMgl9EJMEo+EVEEkyXOI9fRCTu3KGmGmoqoaYKqitbGa6KPR7u8NgLoM8x7fpWFPySOOq+uMGXqzr4gtV9aes91lQGX7zKZl4Pnnt1sHALHiz23OzA+Caf15u2uectLqOl+WhhvrDLCDNfG9933fZrKfiqG/4uDmu46uDfXbPDLQR5TRVxcfTJCn7pILUBWV1ZLwQbBeZBwVn/C9xUsNb/QlY2/GI1NW1T6wwV0lXNLL8y3ltV2ltSKiSlQHIqJCXXG06JPTY3nJoBaT2D4eRg/pQ2DKfUW38Uw8mRbDYF/5HKHSr3wb5PoOwT2Lez0fOdTY/fvzsWknTgndksqdEXNLXhh7/+lyI59cDr3TJjz+t/6eteTz7wvP6X/qBlpTT8Ure4/savp4DV/2J6bLvXbrv6d7c7aLzXe72p5x5yvpbWHXYZYdbd3LSHum6aCeo2DFtSw/8kJDQFf1dQVXEgoOsC+5MmwntnwyCv3t/8MlO7Q2ZvyOgFGb2h32jIyIG0LEju1kSYNgreFsO08euthGmSzjEQ6UgK/o5UUw3lpc2Ed1PPgxCv2NP8MpNSgwDvHXvsPQwyxgfjcg6Mb/w8Ja3j3reIdCoK/rZwj4Vxk3veLQT5vl0024RiSZDe60Aw9+gHfU+sF9g5TQd5t+76d1dEDomCv7K8hXbwRs0n9QO+pQOFaVkHmlAye0POkOb3vDNyYs/TstXkISId4sgO/k1/ga2vNxHquw48r9rX/Pwp6Q1DOve41ptQMnJi7dciIp3UkR387zwGr94dO/Oifkj3Ohr6jzmwt91ckKdmxPsdiIi0uyM7+E+9AT7z3VjTi9rBRUSAIz34M3rFuwIRkU5HRxNFRBKMgl9EJMEo+EVEEoyCX0QkwSj4RUQSjIJfRCTBKPhFRBKMgl9EJMEo+EVEEoyCX0QkwSj4RUQSjIJfRCTBKPhFRBKMgl9EJMEo+EVEEoyCX0QkwSj4RUQSjIJfRCTBKPhFRBJMpMFvZleZ2dtm9ncz+52ZpZtZbzN72sw2BI85UdYgIiINRRb8ZjYQuAKY6O6jgWTgAmAR8Ky7jwCeDYZFRKSDRN3UkwJkmFkKkAlsBc4ClgevLwfOjrgGERGpJ7Lgd/d/AT8BPgSKgFJ3/1/gKHcvCqYpAvo2Nb+ZXWxma8xsTXFxcVRliogknCibenKI7d0PAwYA3c3sS2Hnd/e73X2iu0/My8uLqkwRkYQTZVPPacA/3b3Y3SuBR4EpwHYz6w8QPH4cYQ0iItJIlMH/ITDZzDLNzIDZwLvAE8CCYJoFwOMR1iAiIo2kRLVgd3/FzB4GXgeqgDeAu4EewB/M7KvE/jicG1UNIiJysMiCH8DdbwJuajR6P7G9fxERiQNduSsikmAU/CIiCUbBLyKSYBT8IiIJRsEvIpJgFPwiIglGwS8ikmAU/CIiCUbBLyKSYBT8IiIJRsEvIpJgFPwiIglGwS8ikmBaDX4zG90RhYiISMcIs8d/l5m9ambfNLNeURckIiLRajX43X0aMB84GlhjZg+a2b9FXpmIiEQiVBu/u28Avgt8B5gJLDWz9WZ2TpTFiYhI+wvTxj/GzG4ndr/czwBnuvsJwfPbI65PRETaWZhbL/4MuAdY7O77ake6+1Yz+25klYmISCTCBP/ngX3uXg1gZklAuruXuftvIq1ORETaXZg2/meAjHrDmcE4ERHpgsIEf7q776kdCJ5nRleSiIhEKUzw7zWz8bUDZjYB2NfC9CIi0omFaeO/EnjIzLYGw/2B8yOrSEREItVq8Lv7a2Y2EjgeMGC9u1dGXpmIiEQizB4/xEL/RCAdGGdmuPv90ZUlIiJRaTX4zewmYBax4H8KOB0oBBT8IiJdUJiDu/OA2cA2d18IjAXSIq1KREQiEyb497l7DVBlZlnAx8DwaMsSEZGohGnjXxN0x3wPsBbYA7waZVEiIhKdFoPfzAz4kbvvItYv/5+BLHf/W0cUJyIi7a/Fph53d+CxesObFPoiIl1bmDb+l83spMgrERGRDhGmjf9U4D/MbDOwl9hFXO7uYyKtTEREIhEm+E+PvAoREekwYYLfI69CREQ6TJjg/xOx8DdiXTYMA94DRkVYl4iIRCRMJ2359YeDLpr/I7KKREQkUmHO6mnA3V8HQp3lY2a9zOxhM1tvZu+a2Slm1tvMnjazDcFjziFXLSIibRamk7ar6w0mAeOB4pDLXwL82d3nmVk3YnfuWgw86+63mNkiYBHwnUMrW0RE2irMHn/Pej9pxNr8z2ptpqBfnxnAvQDuXhFcAXwWsDyYbDlw9qEWLSIibRemjf/7bVz2cGL/GSwzs7HE+vn5FnCUuxcFyy4ys75NzWxmFwMXAwwePLiNJYiISGOt7vEH7fC96g3nmNn/hFh2CrFmoV+4+zhiF38tCluYu9/t7hPdfWJeXl7Y2UREpBVhmnrygiYaANx9J9DkXnojW4At7v5KMPwwsT8E282sP0Dw+PEhVSwiIoclzHn81WY22N0/BDCzIYS4qMvdt5nZR2Z2vLu/R+xmLu8EPwuAW4LHx9tcvUgXUFlZyZYtWygvL493KXKESk9PZ9CgQaSmpoaaPkzw3wAUmtkLwfAMgrb3EC4HHgjO6PkAWEjsv4w/mNlXgQ+Bc0MuS6RL2rJlCz179mTo0KHEejoXaT/uzo4dO9iyZQvDhg0LNU+Yg7t/Di7amkzs6t2r3L0kZEHrgIlNvDQ7VHUiR4Dy8nKFvkTGzOjTpw/FxWHPsg93cHcuUOnuT7r7H4ndgvHstpcpkngU+hKlQ/18hTm4e5O7l9YOBAd6bzq0skQknsyMa665pm74Jz/5CTfffHPo+X/961+Tl5dHQUEBo0aNYt68eZSVlbU4z6pVq/jrX//a4jSbNm1i9OjRra7/5ptvJjMzk48/PnAuSI8ePcIV305WrVrFGWecAcATTzzBLbfccsjLWLduHU899VTdcFuXc7jCBH9T04Q5NiAinURaWhqPPvooJSWhWmmbdP7557Nu3TrefvttunXrxooVK1qcPkzwH4rc3Fxuu+22Ns3r7tTU1LRbLXPmzGHRotBnp9dpHPxtXc7hChP8a8zsp2Z2jJkNN7PbiV2MJSJdREpKChdffDG33377Qa9t3ryZ2bNnM2bMGGbPns2HH37Y4rKqqqrYu3cvOTmxbrb++Mc/cvLJJzNu3DhOO+00tm/fzqZNm7jrrru4/fbbKSgoYPXq1Wzfvp25c+cyduxYxo4dW/dHobq6mq9//euMGjWKz372s+zbt6/J9V500UWsWLGCTz755KDXfvrTnzJ69GhGjx7NHXfcAcT+mzjhhBP45je/yfjx41m9ejUjR47ka1/7GqNHj2b+/Pk888wzTJ06lREjRvDqq68C8OqrrzJlyhTGjRvHlClTeO+99w5a369//Wsuu+wyAAoKCup+MjIyeOGFF5pcRkVFBd/73vdYsWIFBQUFrFixosFymvs9fOUrX+GKK65gypQpDB8+nIcffrjF308YYfbcLwduBFYQO7j7v8Clh71mkQT0/T++zTtbd7frMk8ckMVNZ7beS/qll17KmDFjuO666xqMv+yyy7jwwgtZsGAB9913H1dccQWPPfbYQfOvWLGCwsJCioqKOO644zjzzDMBmDZtGi+//DJmxq9+9StuvfVWbrvtNi655BJ69OjBt7/9bSD2H8PMmTNZuXIl1dXV7Nmzh507d7JhwwZ+97vfcc8993DeeefxyCOP8KUvfemg9ffo0YOLLrqIJUuW8P3vH+hQYO3atSxbtoxXXnkFd+fkk09m5syZ5OTk8N5777Fs2TJ+/vOfs2nTJjZu3MhDDz3E3XffzUknncSDDz5IYWEhTzzxBD/84Q957LHHGDlyJC+++CIpKSk888wzLF68mEceeaTZ7bpu3Tog9gfw1ltvZcqUKezbt6/JZfzgBz9gzZo1/OxnPwNif0DC/B6KioooLCxk/fr1zJkzh3nz5rX6+25JmLN6DumKWxHpnLKysrjwwgtZunQpGRkZdeNfeuklHn30UQC+/OUvH/SHodb555/Pz372M9ydSy+9lB//+McsWrSILVu2cP7551NUVERFRUWzpxQ+99xz3H///QAkJyeTnZ3Nzp07GTZsGAUFBQBMmDCBTZs2NfserrjiCgoKChocrygsLGTu3Ll0794dgHPOOYfVq1czZ84chgwZwuTJk+umHTZsGPn5sZ7mR40axezZszEz8vPz69ZbWlrKggUL2LBhA2ZGZWVlC1s1ZsOGDVx77bU899xzpKamsm3btkNeRku/h7PPPpukpCROPPFEtm/f3uqyWhOmd8484DpiN15Jrx3v7p857LWLJJgwe+ZRuvLKKxk/fjwLFy5sdprWzhAxM84880zuvPNOFi1axOWXX87VV1/NnDlzWLVq1SEdNIbY8YdaycnJzTb1APTq1YsvfvGL/PznP68b59789aS1fwyaWldSUlLdcFJSElVVVQDceOONnHrqqaxcuZJNmzYxa9asFuvfu3cv5513Hvfccw8DBgxo0zKaUv/3UL/ult5vWGHa+B8A1hO789b3gU3Aa4e9ZhHpcL179+a8887j3nvvrRs3ZcoUfv/73wPwwAMPMG3atFaXU1hYyDHHHAPE9pAHDhwIwPLly+um6dmzJ59++mnd8OzZs/nFL34BxNr1d+9uW5PX1VdfzS9/+cu6oJ4xYwaPPfYYZWVl7N27l5UrVzJ9+vQ2LRsavp/6TTHNWbhwIQsXLmywzuaW0Xib1NeW30NbhQn+Pu5+L7Fz+V9w94uIXcwlIl3QNddc0+DsnqVLl7Js2TLGjBnDb37zG5YsWdLkfLUHJceMGcMbb7zBjTfeCMROtTz33HOZPn06ubm5ddOfeeaZrFy5su7g7pIlS3j++efJz89nwoQJvP32222qPzc3l7lz57J//34Axo8fz1e+8hUmTZrEySefzNe+9jXGjRvXpmUDXHfddVx//fVMnTqV6urqFqfdvHkzDz/8MPfdd1/dAd41a9Y0u4xTTz2Vd955p+7gbn1hfw/twVr7t8HMXnb3yUGPnEuBrcDD7n5MZFU1MnHiRF+zZk1HrU6kXb377ruccMIJ8S5DjnBNfc7MbK27H9R7Qpizev7TzLKBa4A7gSzgqvYoVEREOl6Ys3qeDJ6WAqdGW46IiETtkG+2LiIiXZuCX0QkwSj4RUQSTJgLuNKA/wMMrT+9u/8gurJERCQqYfb4HwfOAqqI3TC99kdEuojk5GQKCgoYO3Ys48ePb7XXzF27djW4OrY5s2bNorVTrTdt2oSZceedd9aNu+yyy0JdHNWehg4dWnf9wpQpU9q0jB/+8IcNhtu6nHgLE/yD3P18d7/V3W+r/Ym8MhFpNxkZGaxbt44333yTH/3oR1x//fUtTh82+MPq27cvS5YsoaKiok3z116l217a2l104+Bvz26nO1KY4P+rmeVHXomIdIjdu3fXdam8Z88eZs+ezfjx48nPz+fxxx8HYNGiRbz//vsUFBRw7bXXAnDrrbeSn5/P2LFjG/Qh/9BDDzFp0iSOO+44Vq9e3eQ68/LymD17doMuHWqtW7eOyZMnM2bMGObOncvOnTuB2H8TixcvZubMmSxZsoRZs2Zx1VVXMWPGDE444QRee+01zjnnHEaMGMF3v/vduuWdffbZTJgwgVGjRnH33Xc3WU/tTVy+973v1V1xO3DgwLo+jJpaxqJFi9i3bx8FBQXMnz+/wXLcnWuvvZbRo0eTn59fd1XuqlWrmDVrFvPmzWPkyJHMnz+/XfraOWzu3uIP8A5QAbwH/A14C/hba/O158+ECRNcpKt65513Dgw89R33+z7fvj9PfafVGpKSknzs2LF+/PHHe1ZWlq9Zs8bd3SsrK720tNTd3YuLi/2YY47xmpoa/+c//+mjRo06UPZTT/kpp5zie/fudXf3HTt2uLv7zJkz/eqrr3Z39z/96U8+e/bsg9Zdu6wPPvjAjz/+eK+qqvJLL73Uly1b5u7u+fn5vmrVKnd3v/HGG/1b3/pW3bK/8Y1v1C1n5syZft1117m7+x133OH9+/f3rVu3enl5uQ8cONBLSkoa1FZWVuajRo2qGz9kyBAvLi52d/fu3bs3qHHXrl2en59ft12aW0bj+WqHH374YT/ttNO8qqrKt23b5kcffbRv3brVn3/+ec/KyvKPPvrIq6urffLkyb569ermf1GHocHnLACs8SYyNcyVu6dH9ldHRDpEbVMPxLr/vfDCC/n73/+Ou7N48WJefPFFkpKS+Ne//tVkt7/PPPMMCxcuJDMzE4h19lbrnHPOAVrvUnnYsGFMmjSJBx98sG5caWkpu3btYubMmQAsWLCAc889t+71888/v8Ey5syZA0B+fj6jRo2if//+AAwfPpyPPvqIPn36sHTpUlauXAnARx99xIYNG+jTp0+zdbk78+fP56qrrmLChAkAh7yMwsJC/v3f/53k5GSOOuooZs6cyWuvvUZWVhaTJk1i0KBBQOymLZs2bYq0A7Ywwly5u9nMxgK1Xc+tdvc3oy1L5Ah1esffX7WxU045hZKSEoqLi3nqqacoLi5m7dq1pKamMnToUMrLyw+ax92b7a65tsvg5OTkVtviFy9ezLx585gxY0aoWpvrVrl+l8q1w1VVVaxatYpnnnmGl156iczMTGbNmtXk+6nv5ptvZtCgQXXNPG1ZhrfQfNO42+n2Pl7RFq228ZvZt4h1zdw3+PmtmV0edWEiEo3169dTXV1Nnz59KC0tpW/fvqSmpvL888+zefNm4ODugz/72c9y33331d1gvanbH4YxcuRITjzxRJ58MtYTTHZ2Njk5OXXHBn7zm9/U7f23RWlpKTk5OWRmZrJ+/XpefvnlFqd/8sknefrpp1m6dGmoZaSmpjZ5U5UZM2awYsUKqqurKS4u5sUXX2TSpEltfh9RC9PU81XgZI/diQsz+3/AS8Q6bBORLqD2oCTE9k6XL19OcnIy8+fP58wzz2TixIkUFBQwcuRIAPr06cPUqVMZPXo0p59+Oj/+8Y9Zt24dEydOpFu3bnz+858/6AyXsG644YYG3SYvX76cSy65hLKyMoYPH86yZcva/D4/97nPcddddzFmzBiOP/74Bnffasptt93G1q1b60J6zpw53HDDDc0u4+KLL2bMmDGMHz+eBx54oG783Llzeemllxg7dixmxq233kq/fv1Yv359m99LlMJ0y/wWcJK7lwfD6cBr7t5hZ/qoW2bpytQts3SE9u6WeRnwipmtDIbPBu5tfnIREenMwhzc/amZrQKmAQYsdPc3oi5MRESi0Wzwm1mWu+82s97E7rO7qd5rvd29bUd3REQkrlra438QOANYC9Q/EGDB8PAI6xI5orR0OqTI4WrtWG1jzQa/u58RPA47zJpEElp6ejo7duygT58+Cn9pd+7Ojh07SE9PDz1PmG6Zn3X32a2NE5GmDRo0iC1btlBcXBzvUuQIlZ6eXnd1cBgttfGnA5lArpnlEGvigdjN1gccTpEiiSQ1NZVhw/SPs3QeLe3x/wdwJbGQX8uB4N8N/Fe0ZYmISFRaauNfAiwxs8vdXVfpiogcIcKcx3+nmY0GTgTS642/P8rCREQkGmEO7t4EzCIW/E8R66a5EFDwi4h0QWHuwDUPmA1sc/eFwFggreVZRESkswoT/PvcvQaoMrMs4GN08ZaISJcVJvjXmFkv4B5iZ/e8DrwadgVmlmxmb5jZk8FwbzN72sw2BI85bSlcRETaptXgd/dvuvsud78L+DdgQdDkE9a3gHfrDS8CnnX3EcCzwbCIiHSQZoPfzMY3/gF6AynB81aZ2SDgC8Cv6o0+C1gePF9OrJtnERHpIC2d1XNb8JgOTATeJHYR1xjgFWLdNLfmDuA6oGe9cUe5exGAuxeZWd+mZjSzi4GLAQYPHhxiVSIiEkaze/zufqq7nwpsBsa7+0R3nwCMAza2tmAzOwP42N3XtqUwd787WOfEvLy8tixCRESaEOYOXCPd/a3aAXf/u5kVhJhvKjDHzD5P7L+GLDP7LbDdzPoHe/v9iZ0lJCIiHSTMWT3vmtmvzGyWmc00s3toeLC2Se5+vbsPcvehwAXAc+7+JeAJYEEw2QLg8TbWLiIibRBmj38h8A1iZ+cAvAj84jDWeQvwBzP7KvAhcO5hLEtERA6RHeqdW+Jh4sSJvmbNmniXISLSpZjZWnef2Hh8S/3x/8HdzzOzt2h460UA3H1MO9coIiIdoKWmntqmnTM6ohAREekYLfXHX3uu/eaOK0dERKLWUlPPpzTRxEPsIi5396zIqhIRkci0tMffs7nXRESk6wpzOicAQdcK9e/A9WEkFYmISKRavYDLzOaY2Qbgn8ALwCbgvyOuS0REIhLmyt3/C0wG/uHuw4jdjesvkVYlIiKRCRP8le6+A0gysyR3fx4oiLYsERGJSpg2/l1m1oNYVw0PmNnHQFW0ZYmISFTC7PGfBZQBVwF/Bt4HzoyyKBERiU6YPf6LgYfcfQsH7pwlIiJdVJg9/izgf8xstZldamZHRV2UiIhEJ8zN1r/v7qOAS4EBwAtm9kzklYmISCTC7PHX+hjYBuwAmrxProiIdH5hLuD6hpmtAp4FcoGvq0tmEZGuK8zB3SHAle6+LuJaRESkA7Qa/O6+qCMKERGRjnEobfwiInIEUPCLiCQYBb+ISIJR8IuIJBgFv4hIglHwi4gkGAW/iEiCUfCLiCQYBb+ISIJR8IuIJBgFv4hIglHwi4gkGAW/iEiCUfCLiHQyZRVVPL/+Y37wx3coKt3X7ssP0x+/iIhEqLrGeXtrKas3lLB6QzFrN++kstrplpLE9BG59M/OaNf1KfhFROJgy84yCjeUsHpDCX95v4RdZZUAnNA/i4VThzF9RC4nDe1Nempyu69bwS8i0gE+La/kpfd3ULixhMINJXxQsheAo7LSmD3yKKaPyGXqsbnk9UyLvBYFv4hIBKqqa3hzSymrNxRTuKGENz7aRXWNk5GazOThvZk/eQjTR+Qyom8PzKxDa1Pwi4i0A3dn844yVm8soXBDMX99fweflldhBvkDs7lk5nCmHZvH+CG9SEtp/+abQ6HgFxFpo9KySv76fgkvbiihcGMxH30SOwNnYK8MvpDfn+kj8phyTB9yuneLc6UNRRb8ZnY0cD/QD6gB7nb3JWbWG1gBDAU2Aee5+86o6hARaS8VVTW88eFOCjfGwv6tLbuoceiRlsIpx/Th69OHM+3YXIbldu/w5ptDEeUefxVwjbu/bmY9gbVm9jTwFeBZd7/FzBYBi4DvRFiHiEibuDvvF+9h9YbYAdmXPthBWUU1yUnG2EHZXPaZEcwYkcvYo3uRmtx1LouKLPjdvQgoCp5/ambvAgOBs4BZwWTLgVUo+EWkk9ixZz9/eX8Hq/9RTOHGEopKywEY2ieTc8YPZNqxeZxyTB+yM1LjXGnbdUgbv5kNBcYBrwBHBX8UcPciM+vbzDwXAxcDDB48uCPKFJEEVF5ZzdrNO+sunnp7624AstJTmDYil8uPzWP6iFyO7p0Z50rbT+TBb2Y9gEeAK919d9h2L3e/G7gbYOLEiR5dhSKSSNyd9ds+jV08tbGEV/+5g/LKGlKSjPFDcvj2Z49j2og88gdmk5zUedvpD0ekwW9mqcRC/wF3fzQYvd3M+gd7+/2Bj6OsQUTk493lFG6MXSVbuLGE4k/3A3Bs3x5ccNJgZhyXy8nD+tA9LTFOdIzyrB4D7gXedfef1nvpCWABcEvw+HhUNYhIYtpXUc0r/9xR1yXCe9s/BaB3925MOzaXaSNyI+kDp6uI8s/bVODLwFtmti4Yt5hY4P/BzL4KfAicG2ENIpIAamqcd4p282JwleyaTTupqK6hW0oSJw3NYe74kUw7NpcT+2eRdIQ23xyKKM/qKQSa28Kzo1qviCSGrbv2UbihhBeDq2Q/2VsBwMh+PVkwZQjTRuQxaWhvMrrF9yrZzigxGrREpMvbs7+Kl4NOzl7cUMwHxbFOzvJ6pjHr+Ly6Ts769kyPc6Wdn4JfRDql6hrnb1t21V089fqHO6mqcdJTkzh5WB++OGkw00fkcdxRHd/JWVen4BeRTuPDHWWs3ljM6n+U8Nf3S9gddHI2ekA2X58xnOkjcpkwJCfunZx1dQp+EYmb0n2VvPT+gdMsN+8oA2BAdjqnj+7PtKD5pncn6+Ssq1Pwi0iHqayuYd1Hu1j9j2JWbyzhzY9inZx175bMKcf04aKpw5g2IpfhnbyTs65OwS8i7WZfRTVbS/exrbScrbuCx9JytpXuo6i0nA8/KaOsopokg7FH9+KyU49l+nF5FHSxTs66OgW/iISyr6KaotKGYR57DEJ+d3ndfWPr69O9G/2y0xmUk8nk4X2YPLw3pwzPJTuz63Zy1tUp+EWE8spqikrLKSrdR9GucrbtPniPfWcTod67ezf6ZaUzKCeDk4b2pl92OgN6pdMvK4MBvdI5Kis9kpuFy+FR8Isc4corq9lWWn4g2IPH2J56LORrL36qLyczlX7ZGQzITmfCkF70z86gf3Z6LNyzM+iXrVDvqhT8Il3Y/qomQn1XeYNw39FEqPfKTKVfVjoDemUwbnAv+men1wV7/14Z9MtK1xWvRzAFv0gntb+qmu2l++vtpTfcYy/a1XSoZ2ekBkGeztije9E/KxbmteP6ZaeT2U1f/USm375IHFRU1bB9d+M99YYBX7Ln4FDPSk+J7Zn3Sid/YK+6MK8d1y8rPWG6Fpa20ydEpJ1VVtewrbThAdL6Ab91Vzkle/YfNF/P9JS6EB81IKsuzOs3wyjUpT3oU5RAKqtrKNtfzd6KKsoqqthb+zx43Lu/mv1V1fEus0upqKqhKDilsSg4vbFkz3680T3jeqal0C9oPz+hXxb9ex04QDqgVzr9sjPooVCXDqJPWifk7lQ0COlq9u5v9FhRxd79sbAuq6hib0U1ZfuDx4p64+uFe0V1Tbzf2hGpe7fkujb04/v1bHCQdEDQpt4zXeesS+eh4D9M7s7+qppGgdwofGtDuZlgbhzce/dXUVUT/jbDGanJdE9LoXtaMpndUujeLZms4ABfZreG4zPTUuhRO1w3PoXMtGS6d0shLSUJXSkfXkpykvbUpctJqE+su1NeWcOe/VXNhm798Xv2VzW9113RcHx1yJA2g8y6kE4hs1ssbHO6d2NQTjCcdvBj/WA+MC4W4hmpyUfsDaFFJBpHdPAvfXYDK9/4V4PAbtz22pwko8nQzeuZxpBumQ3HpyXTIy2lwV51926N9qrTkklPSdZt30Qk7o7o4O/bM438gdkHNXV0byaY6+9Rx5o8FNIicuQ5ooP/gkmDuWDS4HiXISLSqagfVBGRBKPgFxFJMAp+EZEEo+AXEUkwCn4RkQSj4BcRSTAKfhGRBKPgFxFJMOZh+zCIIzMrBja3cfZcoKQdy2kvquvQqK5Do7oOTWetCw6vtiHuntd4ZJcI/sNhZmvcfWK862hMdR0a1XVoVNeh6ax1QTS1qalHRCTBKPhFRBJMIgT/3fEuoBmq69CorkOjug5NZ60LIqjtiG/jFxGRhhJhj19EROpR8IuIJJgjJvjN7HNm9p6ZbTSzRU28bma2NHj9b2Y2vpPUNcvMSs1sXfDzvQ6o6T4z+9jM/t7M6/HaVq3V1eHbKljv0Wb2vJm9a2Zvm9m3mpimw7dZyLri8flKN7NXzezNoK7vNzFNPLZXmLri8hkL1p1sZm+Y2ZNNvNa+28vdu/wPkAy8DwwHugFvAic2mubzwH8DBkwGXukkdc0Cnuzg7TUDGA/8vZnXO3xbhayrw7dVsN7+wPjgeU/gH53k8xWmrnh8vgzoETxPBV4BJneC7RWmrrh8xoJ1Xw082NT623t7HSl7/JOAje7+gbtXAL8Hzmo0zVnA/R7zMtDLzPp3gro6nLu/CHzSwiTx2FZh6ooLdy9y99eD558C7wIDG03W4dssZF0dLtgGe4LB1OCn8Vkk8dheYeqKCzMbBHwB+FUzk7Tr9jpSgn8g8FG94S0c/AUIM0086gI4Jfj387/NbFTENYURj20VVly3lZkNBcYR21usL67brIW6IA7bLGi2WAd8DDzt7p1ie4WoC+LzGbsDuA6oaeb1dt1eR0rwWxPjGv8lDzNNewuzzteJ9acxFrgTeCzimsKIx7YKI67bysx6AI8AV7r77sYvNzFLh2yzVuqKyzZz92p3LwAGAZPMbHSjSeKyvULU1eHby8zOAD5297UtTdbEuDZvryMl+LcAR9cbHgRsbcM0HV6Xu++u/ffT3Z8CUs0sN+K6WhOPbdWqeG4rM0slFq4PuPujTUwSl23WWl3x/ny5+y5gFfC5Ri/F9TPWXF1x2l5TgTlmtolYc/BnzOy3jaZp1+11pAT/a8AIMxtmZt2AC4AnGk3zBHBhcHR8MlDq7kXxrsvM+pmZBc8nEfud7Ii4rtbEY1u1Kl7bKljnvcC77v7TZibr8G0Wpq54bDMzyzOzXsHzDOA0YH2jyeKxvVqtKx7by92vd/dB7j6UWEY85+5fajRZu26vlLaX23m4e5WZXQb8D7Ezae5z97fN7JLg9buAp4gdGd8IlAELO0ld84BvmFkVsA+4wIPD+FExs98RO3sh18y2ADcRO9AVt20Vsq4O31aBqcCXgbeC9mGAxcDgerXFY5uFqSse26w/sNzMkokF5x/c/cl4fx9D1hWvz9hBotxe6rJBRCTBHClNPSIiEpKCX0QkwSj4RUQSjIJfRCTBKPhFRBKMgl+knVmsh8eDelgU6SwU/CIiCUbBLwnJzL5ksb7Z15nZL4OLejCzPWZ2m5m9bmbPmlleML7AzF62WF/oK80sJxh/rJk9E3Tq9bqZHROsooeZPWxm683sgXpXg95iZu8Ey/lJXN68JDwFvyQcMzsBOB+YGnTYVQ3MD17uDrzu7uOBF4hdPQxwP/Addx8DvFVv/APAfwWdek0Bai+jHwdcCZxI7H4MU82sNzAXGBUs5z+jeo8iLVHwSyKaDUwAXgu6OphNLJwh1i3uiuD5b4FpZpYN9HL3F4Lxy4EZZtYTGOjuKwHcvdzdy4JpXnX3Le5eA6wDhgK7gXLgV2Z2DrFL70U63BHRV4/IITJgubtfH2Lalvo0aaqr3Fr76z2vBlKCvpsmEftDcwFwGfCZEDWItCvt8UsiehaYZ2Z9Acyst5kNCV5LItZRF8AXgUJ3LwV2mtn0YPyXgReCvu+3mNnZwXLSzCyzuZVarN/87KC73yuBgnZ9VyIhaY9fEo67v2Nm3wX+18ySgErgUmAzsBcYZWZrgVJixwIAFgB3BcH+AQd6R/wy8Esz+0GwnHNbWHVP4HEzSyf238JV7fvORMJR75wi9ZjZHnfvEe86RKKkph4RkQSjPX4RkQSjPX4RkQSj4BcRSTAKfhGRBKPgFxFJMAp+EZEE8/8BDzAbc9hwUQoAAAAASUVORK5CYII=\n",
"text/plain": [
"<Figure size 432x288 with 1 Axes>"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"# Plot the diagram to show the accuracy\n",
"\n",
"plt.plot(training_results['validation_accuracy'],label='No Batch Normalization')\n",
"plt.plot(training_results_Norm['validation_accuracy'],label='Batch Normalization')\n",
"plt.ylabel('validation accuracy')\n",
"plt.xlabel('epochs ') \n",
"plt.legend()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<!--Empty Space for separating topics-->\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<a href=\"http://cocl.us/pytorch_link_bottom\">\n",
" <img src=\"https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/DL0110EN/notebook_images%20/notebook_bottom%20.png\" width=\"750\" alt=\"PyTorch Bottom\" />\n",
"</a>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<h2>About the Authors:</h2> \n",
"\n",
"<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a> \n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Change Log\n",
"\n",
"| Date (YYYY-MM-DD) | Version | Changed By | Change Description |\n",
"| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n",
"| 2020-09-23 | 2.0 | Srishti | Migrated Lab to Markdown and added to course repo in GitLab |\n",
"\n",
"<hr>\n",
"\n",
"## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"<hr>\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Copyright © 2018 <a href=\"cognitiveclass.ai?utm_source=bducopyrightlink&utm_medium=dswb&utm_campaign=bdu\">cognitiveclass.ai</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python",
"language": "python",
"name": "conda-env-python-py"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.6.11"
}
},
"nbformat": 4,
"nbformat_minor": 4
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment