Skip to content

Instantly share code, notes, and snippets.

@pdewilde

pdewilde/hw1.ipynb

Created December 9, 2018 21:40
Show Gist options
  • Save pdewilde/eb15b8e85d167d57b4f6d838ad3f14ab to your computer and use it in GitHub Desktop.
Save pdewilde/eb15b8e85d167d57b4f6d838ad3f14ab to your computer and use it in GitHub Desktop.
Download ZIP
yee
Display the source blob
Display the rendered blob
Raw
hw1.ipynb
{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "homework1_colab.ipynb",
"version": "0.3.2",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"metadata": {
"id": "5EJQLxb5_Zh8",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Homework 1\n",
"In this homework, we will have you train some convolutional neural networks! We will start with a small dataset, and then work our way up to ImageNet!\n",
"\n",
"For this homework we will train a classifier for Tiny Imagenet which you can read about here https://tiny-imagenet.herokuapp.com/ but don't bother downloading it just yet.\n",
"\n",
"A note on file paths: Last homework, we used the homework1 folder. That was a mistake as it was technically homework 0. You may want to rename the folder from last time to homework0 before you begin.\n",
"\n",
"# Part 0: Initial Setup\n",
"You should recognize this code from last time.\n"
]
},
{
"metadata": {
"id": "BkEgmb7Odtec",
"colab_type": "code",
"outputId": "1d769bac-a871-4977-ca35-429e0e15a21c",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 360
}
},
"cell_type": "code",
"source": [
"# This is code to download and install pytorch\n",
"import os\n",
"from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
"platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
"\n",
"accelerator = 'cu80' if os.path.exists('/opt/bin/nvidia-smi') else 'cpu'\n",
"!pip install http://download.pytorch.org/whl/{accelerator}/torch-0.4.1-{platform}-linux_x86_64.whl torchvision\n",
"\n",
"import torch\n",
"print('Version', torch.__version__)\n",
"print('CUDA enabled:', torch.cuda.is_available())"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting torch==0.4.1 from http://download.pytorch.org/whl/cu80/torch-0.4.1-cp36-cp36m-linux_x86_64.whl\n",
"\u001b[?25l Downloading http://download.pytorch.org/whl/cu80/torch-0.4.1-cp36-cp36m-linux_x86_64.whl (483.0MB)\n",
"\u001b[K 100% |████████████████████████████████| 483.0MB 62.9MB/s \n",
"tcmalloc: large alloc 1073750016 bytes == 0x56190000 @ 0x7f0a4e9262a4 0x594e17 0x626104 0x51190a 0x4f5277 0x510c78 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f6070 0x510c78 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f6070 0x4f3338 0x510fb0 0x5119bd 0x4f6070\n",
"\u001b[?25hCollecting torchvision\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/ca/0d/f00b2885711e08bd71242ebe7b96561e6f6d01fdb4b9dcf4d37e2e13c5e1/torchvision-0.2.1-py2.py3-none-any.whl (54kB)\n",
"\u001b[K 100% |████████████████████████████████| 61kB 2.4MB/s \n",
"\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.14.6)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.11.0)\n",
"Collecting pillow>=4.1.1 (from torchvision)\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/62/94/5430ebaa83f91cc7a9f687ff5238e26164a779cca2ef9903232268b0a318/Pillow-5.3.0-cp36-cp36m-manylinux1_x86_64.whl (2.0MB)\n",
"\u001b[K 100% |████████████████████████████████| 2.0MB 4.5MB/s \n",
"\u001b[?25hInstalling collected packages: torch, pillow, torchvision\n",
" Found existing installation: Pillow 4.0.0\n",
" Uninstalling Pillow-4.0.0:\n",
" Successfully uninstalled Pillow-4.0.0\n",
"Successfully installed pillow-5.3.0 torch-0.4.1 torchvision-0.2.1\n",
"Version 0.4.1\n",
"CUDA enabled: True\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "gptyTwi0AQHf",
"colab_type": "code",
"outputId": "56e45e1e-d3cb-4867-d0c4-def7abdd9127",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
}
},
"cell_type": "code",
"source": [
"from google.colab import drive\n",
"drive.mount('/gdrive')\n",
"!ls /gdrive"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n",
"\n",
"Enter your authorization code:\n",
"··········\n",
"Mounted at /gdrive\n",
"'My Drive' 'Team Drives'\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "BjUlLqslACEW",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 1: Upload the Dataset\n",
"Run this code to navigate to the BASE_PATH directory and upload the homework1.tar file inside the BASE_PATH, then extract it.\n",
"\n",
"Have a look at pt_util. We moved some of the useful functions out of the python notebook to make it less cluttered, and added a few more useful functions.\n",
"\n",
"I made the BASE_PATH and DATA_PATH variables so you don't have to copy the same strings all over the place if you want to move the locations of the files around.\n"
]
},
{
"metadata": {
"id": "2UtsxBCpChPn",
"colab_type": "code",
"outputId": "1bd79e9c-4d61-478a-95ba-817126c807e8",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
}
},
"cell_type": "code",
"source": [
"BASE_PATH = '/gdrive/My Drive/colab_files/homework1/'\n",
"if not os.path.exists(BASE_PATH):\n",
" os.makedirs(BASE_PATH)\n",
"DATA_PATH = BASE_PATH + 'tiny_imagenet/'\n",
"\n",
"!pwd\n",
"!ls\n",
"os.chdir(BASE_PATH)\n",
"if not os.path.exists(DATA_PATH + 'train.h5'):\n",
" !wget http://pjreddie.com/media/files/homework1.tar\n",
" !tar -xvf homework1.tar\n",
" !rm homework1.tar\n",
"os.chdir('/content')"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"/content\n",
"sample_data\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "yvK-kdPRav5L",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"import torch\n",
"import torch.nn as nn\n",
"from torchvision import datasets\n",
"from torchvision import transforms\n",
"import numpy as np\n",
"import os\n",
"import torch.nn.functional as F\n",
"import torch.optim as optim\n",
"import h5py\n",
"import sys\n",
"sys.path.append(BASE_PATH)\n",
"import pt_util"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "b8NWTxZvJeAE",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 2: Defining the Network\n",
"We're giving you no instructions on this part. Welcome to deep learning research! See if you can get above 40% accuracy. You probably want to use the Cross Entropy error again, but who knows, maybe you can find a better loss function. We will give you a few hints of things to try. Have a look at https://github.com/pytorch/examples/blob/master/mnist/main.py for a basic neural network implementation.\n",
"\n",
"\n",
"- Activation functions other than ReLU\n",
"- Batch Norm\n",
"- Dropout\n",
"- Residual connections\n",
"\n",
"Additionally, the current saving function always saves the latest results. You may want to modify it to only save the results if they have the highest test accuracy. Please make this modification.\n",
"\n"
]
},
{
"metadata": {
"id": "RA6lPT8Ceubk",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"\n",
"\n",
"class TinyImagenetNet(nn.Module):\n",
" def __init__(self):\n",
" super(TinyImagenetNet, self).__init__()\n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # now just using a sample of darknet\n",
" self.conv4 = nn.Conv2d(3, 128, kernel_size=3, stride=1)\n",
" self.conv5 = nn.Conv2d(128, 64, kernel_size=1, stride=1)\n",
" self.conv6 = nn.Conv2d(64, 128, kernel_size=3, stride=1)\n",
" self.pool7 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv8 = nn.Conv2d(128, 256, kernel_size=3, stride = 1)\n",
" self.conv9 = nn.Conv2d(256, 128, kernel_size=1, {
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "homework1_colab.ipynb",
"version": "0.3.2",
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
},
"accelerator": "GPU"
},
"cells": [
{
"metadata": {
"id": "5EJQLxb5_Zh8",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Homework 1\n",
"In this homework, we will have you train some convolutional neural networks! We will start with a small dataset, and then work our way up to ImageNet!\n",
"\n",
"For this homework we will train a classifier for Tiny Imagenet which you can read about here https://tiny-imagenet.herokuapp.com/ but don't bother downloading it just yet.\n",
"\n",
"A note on file paths: Last homework, we used the homework1 folder. That was a mistake as it was technically homework 0. You may want to rename the folder from last time to homework0 before you begin.\n",
"\n",
"# Part 0: Initial Setup\n",
"You should recognize this code from last time.\n"
]
},
{
"metadata": {
"id": "BkEgmb7Odtec",
"colab_type": "code",
"outputId": "1d769bac-a871-4977-ca35-429e0e15a21c",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 360
}
},
"cell_type": "code",
"source": [
"# This is code to download and install pytorch\n",
"import os\n",
"from wheel.pep425tags import get_abbr_impl, get_impl_ver, get_abi_tag\n",
"platform = '{}{}-{}'.format(get_abbr_impl(), get_impl_ver(), get_abi_tag())\n",
"\n",
"accelerator = 'cu80' if os.path.exists('/opt/bin/nvidia-smi') else 'cpu'\n",
"!pip install http://download.pytorch.org/whl/{accelerator}/torch-0.4.1-{platform}-linux_x86_64.whl torchvision\n",
"\n",
"import torch\n",
"print('Version', torch.__version__)\n",
"print('CUDA enabled:', torch.cuda.is_available())"
],
"execution_count": 1,
"outputs": [
{
"output_type": "stream",
"text": [
"Collecting torch==0.4.1 from http://download.pytorch.org/whl/cu80/torch-0.4.1-cp36-cp36m-linux_x86_64.whl\n",
"\u001b[?25l Downloading http://download.pytorch.org/whl/cu80/torch-0.4.1-cp36-cp36m-linux_x86_64.whl (483.0MB)\n",
"\u001b[K 100% |████████████████████████████████| 483.0MB 62.9MB/s \n",
"tcmalloc: large alloc 1073750016 bytes == 0x56190000 @ 0x7f0a4e9262a4 0x594e17 0x626104 0x51190a 0x4f5277 0x510c78 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f6070 0x510c78 0x5119bd 0x4f5277 0x4f3338 0x510fb0 0x5119bd 0x4f6070 0x4f3338 0x510fb0 0x5119bd 0x4f6070\n",
"\u001b[?25hCollecting torchvision\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/ca/0d/f00b2885711e08bd71242ebe7b96561e6f6d01fdb4b9dcf4d37e2e13c5e1/torchvision-0.2.1-py2.py3-none-any.whl (54kB)\n",
"\u001b[K 100% |████████████████████████████████| 61kB 2.4MB/s \n",
"\u001b[?25hRequirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.14.6)\n",
"Requirement already satisfied: six in /usr/local/lib/python3.6/dist-packages (from torchvision) (1.11.0)\n",
"Collecting pillow>=4.1.1 (from torchvision)\n",
"\u001b[?25l Downloading https://files.pythonhosted.org/packages/62/94/5430ebaa83f91cc7a9f687ff5238e26164a779cca2ef9903232268b0a318/Pillow-5.3.0-cp36-cp36m-manylinux1_x86_64.whl (2.0MB)\n",
"\u001b[K 100% |████████████████████████████████| 2.0MB 4.5MB/s \n",
"\u001b[?25hInstalling collected packages: torch, pillow, torchvision\n",
" Found existing installation: Pillow 4.0.0\n",
" Uninstalling Pillow-4.0.0:\n",
" Successfully uninstalled Pillow-4.0.0\n",
"Successfully installed pillow-5.3.0 torch-0.4.1 torchvision-0.2.1\n",
"Version 0.4.1\n",
"CUDA enabled: True\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "gptyTwi0AQHf",
"colab_type": "code",
"outputId": "56e45e1e-d3cb-4867-d0c4-def7abdd9127",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 139
}
},
"cell_type": "code",
"source": [
"from google.colab import drive\n",
"drive.mount('/gdrive')\n",
"!ls /gdrive"
],
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"text": [
"Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob&scope=email%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdocs.test%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.photos.readonly%20https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fpeopleapi.readonly&response_type=code\n",
"\n",
"Enter your authorization code:\n",
"··········\n",
"Mounted at /gdrive\n",
"'My Drive' 'Team Drives'\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "BjUlLqslACEW",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 1: Upload the Dataset\n",
"Run this code to navigate to the BASE_PATH directory and upload the homework1.tar file inside the BASE_PATH, then extract it.\n",
"\n",
"Have a look at pt_util. We moved some of the useful functions out of the python notebook to make it less cluttered, and added a few more useful functions.\n",
"\n",
"I made the BASE_PATH and DATA_PATH variables so you don't have to copy the same strings all over the place if you want to move the locations of the files around.\n"
]
},
{
"metadata": {
"id": "2UtsxBCpChPn",
"colab_type": "code",
"outputId": "1bd79e9c-4d61-478a-95ba-817126c807e8",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 51
}
},
"cell_type": "code",
"source": [
"BASE_PATH = '/gdrive/My Drive/colab_files/homework1/'\n",
"if not os.path.exists(BASE_PATH):\n",
" os.makedirs(BASE_PATH)\n",
"DATA_PATH = BASE_PATH + 'tiny_imagenet/'\n",
"\n",
"!pwd\n",
"!ls\n",
"os.chdir(BASE_PATH)\n",
"if not os.path.exists(DATA_PATH + 'train.h5'):\n",
" !wget http://pjreddie.com/media/files/homework1.tar\n",
" !tar -xvf homework1.tar\n",
" !rm homework1.tar\n",
"os.chdir('/content')"
],
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"text": [
"/content\n",
"sample_data\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "yvK-kdPRav5L",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"import torch\n",
"import torch.nn as nn\n",
"from torchvision import datasets\n",
"from torchvision import transforms\n",
"import numpy as np\n",
"import os\n",
"import torch.nn.functional as F\n",
"import torch.optim as optim\n",
"import h5py\n",
"import sys\n",
"sys.path.append(BASE_PATH)\n",
"import pt_util"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "b8NWTxZvJeAE",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 2: Defining the Network\n",
"We're giving you no instructions on this part. Welcome to deep learning research! See if you can get above 40% accuracy. You probably want to use the Cross Entropy error again, but who knows, maybe you can find a better loss function. We will give you a few hints of things to try. Have a look at https://github.com/pytorch/examples/blob/master/mnist/main.py for a basic neural network implementation.\n",
"\n",
"\n",
"- Activation functions other than ReLU\n",
"- Batch Norm\n",
"- Dropout\n",
"- Residual connections\n",
"\n",
"Additionally, the current saving function always saves the latest results. You may want to modify it to only save the results if they have the highest test accuracy. Please make this modification.\n",
"\n"
]
},
{
"metadata": {
"id": "RA6lPT8Ceubk",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"\n",
"\n",
"class TinyImagenetNet(nn.Module):\n",
" def __init__(self):\n",
" super(TinyImagenetNet, self).__init__()\n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # now just using a sample of darknet\n",
" self.conv4 = nn.Conv2d(3, 128, kernel_size=3, stride=1)\n",
" self.conv5 = nn.Conv2d(128, 64, kernel_size=1, stride=1)\n",
" self.conv6 = nn.Conv2d(64, 128, kernel_size=3, stride=1)\n",
" self.pool7 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv8 = nn.Conv2d(128, 256, kernel_size=3, stride = 1)\n",
" self.conv9 = nn.Conv2d(256, 128, kernel_size=1, stride=1)\n",
" self.conv10 = nn.Conv2d(128, 256, kernel_size=3, stride=1)\n",
" self.pool11 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv16 = nn.Conv2d(256, 512, kernel_size=3, stride=1)\n",
" self.pool17 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.avg4b4 = nn.AvgPool2d(kernel_size=5, stride=1, padding=0)\n",
" self.fc = nn.Linear(512, 256)\n",
" \n",
" \n",
" #use for save_best_model\n",
" self.best = 0.0\n",
" \n",
" def forward(self, x):# TODO define the forward pass\n",
" #raise NotImplementedError('Need to define the forward pass')\n",
" # dimms with batch size 10\n",
" x = x # [10, 3, 64, 64]\n",
" x = F.relu(self.conv4(x)) # [10, 128, 62, 62]\n",
" x = F.relu(self.conv5(x)) # [10, 64, 62, 62]\n",
" x = F.relu(self.conv6(x)) # [10, 128, 60, 60]\n",
" x = self.pool7(x) # [10, 128, 30, 30]\n",
" x = F.relu(self.conv8(x)) # [10, 256, 28, 28]\n",
" x = F.relu(self.conv9(x)) #[10, 128, 28, 28]\n",
" x = F.relu(self.conv10(x)) # [10, 256, 26, 26]\n",
" x = self.pool11(x) # [10, 256, 13, 13]\n",
" x = F.relu(self.conv16(x)) # [10, 512, 11, 11]\n",
" x = self.pool17(x) # [10, 512, 5, 5]\n",
" x = self.avg4b4(x) # [10, 512, 1, 1]\n",
" x = x.view(-1, 512)\n",
" x = self.fc(x)\n",
" \n",
"\n",
" return F.log_softmax(x, dim=1)\n",
"\n",
" def loss(self, prediction, label, reduction='elementwise_mean'):\n",
" loss_val = F.cross_entropy(prediction, label.squeeze(), reduction=reduction)\n",
" return loss_val\n",
"\n",
" def save_model(self, file_path, num_to_keep=1):\n",
" pt_util.save(self, file_path, num_to_keep)\n",
" \n",
" def save_best_model(self, accuracy, file_path, num_to_keep=1):\n",
" # TODO save the model if it is the best\n",
" if (accuracy > self.best):\n",
" self.best = accuracy;\n",
" pt_util.save(self, file_path, num_to_keep)\n",
" else:\n",
" print('Old model was better new: ' + str(accuracy) + 'old: ' + str(self.best))\n",
" #raise NotImplementedError('Need to implement save_best_model')\n",
"\n",
" def load_model(self, file_path):\n",
" pt_util.restore(self, file_path)\n",
"\n",
" def load_last_model(self, dir_path):\n",
" return pt_util.restore_latest(self, dir_path)\n",
" \n",
" \n"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "yiJSkXjiKpDL",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"This time we are giving you the train and test functions, but feel free to modify them if you want. \n",
"\n",
"You may need to return some additional information for the logging portion of this assignment.\n"
]
},
{
"metadata": {
"id": "pmuzixXrkuYs",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"import time\n",
"def train(model, device, train_loader, optimizer, epoch, log_interval):\n",
" model.train()\n",
" for batch_idx, (data, label) in enumerate(train_loader):\n",
" data, label = data.to(device), label.to(device)\n",
" optimizer.zero_grad()\n",
" output = model(data)\n",
" loss = model.loss(output, label)\n",
" loss.backward()\n",
" optimizer.step()\n",
" if batch_idx % log_interval == 0:\n",
" print('{} Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
" time.ctime(time.time()),\n",
" epoch, batch_idx * len(data), len(train_loader.dataset),\n",
" 100. * batch_idx / len(train_loader), loss.item()))\n",
"\n",
"def test(model, device, test_loader, return_images=False, log_interval=None):\n",
" model.eval()\n",
" test_loss = 0\n",
" correct = 0\n",
"\n",
" correct_images = []\n",
" correct_values = []\n",
"\n",
" error_images = []\n",
" predicted_values = []\n",
" gt_values = []\n",
" with torch.no_grad():\n",
" for batch_idx, (data, label) in enumerate(test_loader):\n",
" data, label = data.to(device), label.to(device)\n",
" output = model(data)\n",
" test_loss_on = model.loss(output, label, reduction='sum').item()\n",
" test_loss += test_loss_on\n",
" pred = output.max(1)[1]\n",
" correct_mask = pred.eq(label.view_as(pred))\n",
" num_correct = correct_mask.sum().item()\n",
" correct += num_correct\n",
" if return_images:\n",
" if num_correct > 0:\n",
" correct_images.append(data[correct_mask, ...].data.cpu().numpy())\n",
" correct_value_data = label[correct_mask].data.cpu().numpy()[:, 0]\n",
" correct_values.append(correct_value_data)\n",
" if num_correct < len(label):\n",
" error_data = data[~correct_mask, ...].data.cpu().numpy()\n",
" error_images.append(error_data)\n",
" predicted_value_data = pred[~correct_mask].data.cpu().numpy()\n",
" predicted_values.append(predicted_value_data)\n",
" gt_value_data = label[~correct_mask].data.cpu().numpy()[:, 0]\n",
" gt_values.append(gt_value_data)\n",
" if log_interval is not None and batch_idx % log_interval == 0:\n",
" print('{} Test: [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
" time.ctime(time.time()),\n",
" batch_idx * len(data), len(test_loader.dataset),\n",
" 100. * batch_idx / len(test_loader), test_loss_on))\n",
" if return_images:\n",
" correct_images = np.concatenate(correct_images, axis=0)\n",
" error_images = np.concatenate(error_images, axis=0)\n",
" predicted_values = np.concatenate(predicted_values, axis=0)\n",
" correct_values = np.concatenate(correct_values, axis=0)\n",
" gt_values = np.concatenate(gt_values, axis=0)\n",
"\n",
" test_loss /= len(test_loader.dataset)\n",
" test_accuracy = 100. * correct / len(test_loader.dataset)\n",
"\n",
" print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
" test_loss, correct, len(test_loader.dataset), test_accuracy))\n",
" if return_images:\n",
" return correct_images, correct_values, error_images, predicted_values, gt_values, test_accuracy"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "EwMDBwoCDRS_",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# PART 3: Loading Data\n",
"PyTorch has a nice interface for dealing with a variety of data. You can read a good tutorial here https://pytorch.org/tutorials/beginner/data_loading_tutorial.html\n",
"Your friendly neighborhood TAs have made it even easier by preprocessing the data into a nice format. The data you uploaded is stored using hdf5 files which can be acecces a lot like Numpy arrays using the h5py package. In each of the files, there is a \"dataset\" called 'images', and one called 'labels'. Read more about h5py here http://docs.h5py.org/en/latest/quick.html\n",
"\n",
"Speed hint: With small datasets, it is almost always a good idea to cache the data to disk rather than continually read from files.\n"
]
},
{
"metadata": {
"id": "igPTIlBtk2vu",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"\n",
"# Data loader\n",
"class H5Dataset(torch.utils.data.Dataset):\n",
" def __init__(self, h5_file, transform=None):\n",
" \n",
" # Implement data loading.\n",
" f = h5py.File(h5_file, 'r')\n",
" self.images = f['images'].value\n",
" self.labels = f['labels'].value\n",
" \n",
" #self.labels = np.array(self.hfile['labels'])\n",
" self.transform = transform\n",
" #raise NotImplementedError('Need to implement the data loading')\n",
" f.close()\n",
"\n",
" def __len__(self):\n",
" # Implement the length function\n",
" return self.images.shape[0]\n",
" \n",
"\n",
" def __getitem__(self, idx):\n",
" # implement the getitem function\n",
" # You should return a tuple of:\n",
" # a torch tensor containing single image in CxHxW format and\n",
" # the label as a single tensor scalar\n",
" #raise NotImplementedError('Need to implement the data loading')\n",
" \n",
" # best practice f = h5py.File() and then f[\"image\"][idx] in here\n",
" \n",
" # w and h may be swapped, but channel is first\n",
" data = torch.FloatTensor(self.images[idx]).permute(2, 0, 1)\n",
" label = torch.LongTensor(self.labels[idx])\n",
" if self.transform:\n",
" data = self.transform(data)\n",
" return (data, label)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "tfWvYzBaKcO7",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 4: Dataset Augmentation\n",
"In the MNIST assignment, we didn't do any data augmentation because MNIST is kind of easy.\n",
"\n",
"In this assignment, you may find that data augmentation helps you a lot (or possibly hurts your performance).\n",
"\n",
"You can find a bunch preimplemented here https://pytorch.org/docs/stable/torchvision/transforms.html and you can also do your own as seen in the tutorial from part 3.\n",
"\n",
"Play around with various data augmentations we will suggest some.\n",
"\n",
"- ToPILImage - This one is useful for a lot of the built in transforms which expect PIL images. \n",
"- RandomHorizontalFlip\n",
"- RandomResizedCrop\n",
"- ColorJitter\n",
"- RandomRotation\n",
"- Normalize\n",
"- Adding various types of noise\n",
"- ToTensor - PyTorch expects the output from the dataset to be a tensor in CxHxW format.\n",
"\n",
"\n",
"Note: You should be careful about which of these you apply to the test data. You usually don't want to apply noise to the test data, but you do want to normalize it in the same way for example.\n"
]
},
{
"metadata": {
"id": "-5JeXSx9LIx3",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"import torchvision\n",
"\n",
"train_transforms = None #transforms.Compose([transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1), transforms.ToTensor(),])\n",
"\n",
"test_transforms = None\n",
"\n",
"data_train = H5Dataset(DATA_PATH + 'train.h5', transform=train_transforms)\n",
"data_test = H5Dataset(DATA_PATH + 'val.h5', transform=test_transforms)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "piz_PoP-N5mK",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 5: Training the network\n",
"Generally, it is useful to see how your training is going. Often people print their loss to make sure it goes down and their accuracy to make sure it goes up. But pictures are better than words. So for this part, you should record and plot the training loss, test loss, and test accuracy (and whatever else you want). \n",
"\n",
"We have created a very simple logging interface which essentially just saves and restores files via pickle in pt_util. Saving and restoring log data is important if you end your run early and want to continue where you left off rather than starting over.\n",
"\n",
"We have also provided a plot function which can plot a single line graph. You can use it and plot each value independently, or change it to plot them all in one graph. \n",
"\n",
"\n",
"__Important note: Do not forget to title your graphs and label your axes. Plots are meaningless without a way to read them.__\n",
"\n",
"Second Note: It will be helpful for you when deciding what network structure, data augmentation, and such work to title the graphs accordingly so you remember.\n",
"Third Note: The default setup right now saves and restores the network weights from a single folder. When you modify network architectures, you may want to save the resulting files in different folders (with appropriate names).\n",
"\n",
"We also provided a function for showing some results, because it's not satisfying to train a neural net, you also want to see what it can do! This can also be useful for figuring out what your network is doing well, and what it is failing at. This type of error analysis is very common when training neural networks.\n"
]
},
{
"metadata": {
"id": "Hj-JBTfwk-4A",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Play around with these constants, you may find a better setting.\n",
"BATCH_SIZE = 256\n",
"TEST_BATCH_SIZE = 10\n",
"EPOCHS = 200\n",
"LEARNING_RATE = 0.01\n",
"MOMENTUM = 0.9\n",
"USE_CUDA = True\n",
"PRINT_INTERVAL = 100\n",
"WEIGHT_DECAY = 0.0005\n",
"LOG_PATH = DATA_PATH + 'log.pkl'\n",
"\n",
"# Now the actual training code\n",
"use_cuda = USE_CUDA and torch.cuda.is_available()\n",
"\n",
"device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
"print('Using device', device)\n",
"import multiprocessing\n",
"print('num cpus:', multiprocessing.cpu_count())\n",
"\n",
"kwargs = {'num_workers': multiprocessing.cpu_count(),\n",
" 'pin_memory': True} if use_cuda else {}\n",
"\n",
"class_names = [line.strip().split(', ') for line in open(DATA_PATH + 'class_names.txt')]\n",
"name_to_class = {line[1]: line[0] for line in class_names}\n",
"class_names = [line[1] for line in class_names]\n",
"\n",
"train_loader = torch.utils.data.DataLoader(data_train, batch_size=BATCH_SIZE,\n",
" shuffle=True, **kwargs)\n",
"test_loader = torch.utils.data.DataLoader(data_test, batch_size=TEST_BATCH_SIZE,\n",
" shuffle=False, **kwargs)\n",
"\n",
"model = TinyImagenetNet().to(device)\n",
"optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n",
"start_epoch = model.load_last_model(DATA_PATH + 'checkpoints')\n",
"\n",
"# You may want to define another default for your log data depending on how you save it.\n",
"log_data = pt_util.read_log(LOG_PATH, [])\n",
"\n",
"correct_images, correct_val, error_images, predicted_val, gt_val = test(model, device, test_loader, True)\n",
"correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
"error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
"pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
"pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])\n",
"\n",
"try:\n",
" for epoch in range(start_epoch, EPOCHS + 1):\n",
" train(model, device, train_loader, optimizer, epoch, PRINT_INTERVAL)\n",
" correct_images, correct_val, error_images, predicted_val, gt_val = test(model, device, test_loader, True)\n",
" # TODO define other things to do at the end of each loop like logging and saving the best model.\n",
"\n",
"\n",
"except KeyboardInterrupt as ke:\n",
" print('Interrupted')\n",
"except:\n",
" import traceback\n",
" traceback.print_exc()\n",
"finally:\n",
" # Always save the most recent model, but don't delete any existing ones.\n",
" model.save_model(DATA_PATH + 'checkpoints/%03d.pt' % epoch, 0)\n",
" \n",
" # Show some current correct/incorrect images.\n",
" correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
" error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
" pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
" pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])"
]
},
{
"metadata": {
"id": "Al5ltOeHvJdz",
"colab_type": "code",
"outputId": "b1a5c5ea-26fe-43e1-852e-480c39714ee1",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 3871
}
},
"cell_type": "code",
"source": [
"# Play around with these constants, you may find a better setting.\n",
"BATCH_SIZE = 256 # stock\n",
"#BATCH_SIZE = 1024 this caused an error\n",
"TEST_BATCH_SIZE = 10\n",
"EPOCHS = 10\n",
"#LEARNING_RATE = 0.01\n",
"LEARNING_RATE = 0.02\n",
"MOMENTUM = 0.9\n",
"USE_CUDA = True\n",
"PRINT_INTERVAL = 100\n",
"WEIGHT_DECAY = 0.0005\n",
"LOG_PATH = DATA_PATH + 'log.pkl'\n",
"\n",
"# Now the actual training code\n",
"use_cuda = USE_CUDA and torch.cuda.is_available()\n",
"\n",
"device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
"print('Using device', device)\n",
"import multiprocessing\n",
"print('num cpus:', multiprocessing.cpu_count())\n",
"\n",
"kwargs = {'num_workers': multiprocessing.cpu_count(),\n",
" 'pin_memory': True} if use_cuda else {}\n",
"\n",
"class_names = [line.strip().split(', ') for line in open(DATA_PATH + 'class_names.txt')]\n",
"name_to_class = {line[1]: line[0] for line in class_names}\n",
"class_names = [line[1] for line in class_names]\n",
"\n",
"train_loader = torch.utils.data.DataLoader(data_train, batch_size=BATCH_SIZE,\n",
" shuffle=True, **kwargs)\n",
"test_loader = torch.utils.data.DataLoader(data_test, batch_size=TEST_BATCH_SIZE,\n",
" shuffle=False, **kwargs)\n",
"\n",
"model = TinyImagenetNet().to(device)\n",
"optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n",
"start_epoch = model.load_last_model(DATA_PATH + 'checkpoints')\n",
"\n",
"# You may want to define another default for your log data depending on how you save it.\n",
"log_data = pt_util.read_log(LOG_PATH, [])\n",
"\n",
"correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy = test(model, device, test_loader, True)\n",
"correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
"error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
"pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
"print(min(predicted_val))\n",
"print(max(predicted_val))\n",
"print(min(gt_val))\n",
"print(max(gt_val))\n",
"print(len(class_names))\n",
"pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])\n",
"\n",
"try:\n",
" for epoch in range(start_epoch, EPOCHS + 1):\n",
" train(model, device, train_loader, optimizer, epoch, PRINT_INTERVAL)\n",
" correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy = test(model, device, test_loader, True)\n",
" # TODO define other things to do at the end of each loop like logging and saving the best model.\n",
" model.save_best_model(test_accuracy, DATA_PATH + 'checkpoints/best.pt')\n",
" \n",
"\n",
"except KeyboardInterrupt as ke:\n",
" print('Interrupted')\n",
"except:\n",
" import traceback\n",
" traceback.print_exc()\n",
"finally:\n",
" # Always save the most recent model, but don't delete any existing ones.\n",
" model.save_model(DATA_PATH + 'checkpoints/%03d.pt' % epoch, 0)\n",
" \n",
" # Show some current correct/incorrect images.\n",
" correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
" error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
" pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
" pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])"
],
"execution_count": 55,
"outputs": [
{
"output_type": "stream",
"text": [
"Using device cuda\n",
"num cpus: 2\n",
"\n",
"Test set: Average loss: 5.5672, Accuracy: 48/10000 (0%)\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe11b830278>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "stream",
"text": [
"123\n",
"173\n",
"0\n",
"199\n",
"200\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe1192cbef0>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "stream",
"text": [
"Tue Oct 30 23:00:32 2018 Train Epoch: 0 [0/100000 (0%)]\tLoss: 5.549253\n",
"Tue Oct 30 23:01:56 2018 Train Epoch: 0 [25600/100000 (26%)]\tLoss: 5.179053\n",
"Tue Oct 30 23:03:20 2018 Train Epoch: 0 [51200/100000 (51%)]\tLoss: 5.062684\n",
"Tue Oct 30 23:04:43 2018 Train Epoch: 0 [76800/100000 (77%)]\tLoss: 4.937111\n",
"\n",
"Test set: Average loss: 4.8247, Accuracy: 340/10000 (3%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:06:21 2018 Train Epoch: 1 [0/100000 (0%)]\tLoss: 4.876338\n",
"Tue Oct 30 23:07:45 2018 Train Epoch: 1 [25600/100000 (26%)]\tLoss: 4.729456\n",
"Tue Oct 30 23:09:09 2018 Train Epoch: 1 [51200/100000 (51%)]\tLoss: 4.511452\n",
"Tue Oct 30 23:10:32 2018 Train Epoch: 1 [76800/100000 (77%)]\tLoss: 4.484566\n",
"\n",
"Test set: Average loss: 4.2662, Accuracy: 1017/10000 (10%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:12:10 2018 Train Epoch: 2 [0/100000 (0%)]\tLoss: 4.280207\n",
"Tue Oct 30 23:13:34 2018 Train Epoch: 2 [25600/100000 (26%)]\tLoss: 4.055193\n",
"Tue Oct 30 23:14:58 2018 Train Epoch: 2 [51200/100000 (51%)]\tLoss: 3.978329\n",
"Tue Oct 30 23:16:22 2018 Train Epoch: 2 [76800/100000 (77%)]\tLoss: 4.068361\n",
"\n",
"Test set: Average loss: 3.8912, Accuracy: 1558/10000 (16%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:18:01 2018 Train Epoch: 3 [0/100000 (0%)]\tLoss: 3.833221\n",
"Tue Oct 30 23:19:24 2018 Train Epoch: 3 [25600/100000 (26%)]\tLoss: 3.750987\n",
"Tue Oct 30 23:20:48 2018 Train Epoch: 3 [51200/100000 (51%)]\tLoss: 3.846452\n",
"Tue Oct 30 23:22:12 2018 Train Epoch: 3 [76800/100000 (77%)]\tLoss: 3.569184\n",
"\n",
"Test set: Average loss: 3.7381, Accuracy: 1882/10000 (19%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:23:50 2018 Train Epoch: 4 [0/100000 (0%)]\tLoss: 3.777778\n",
"Tue Oct 30 23:25:14 2018 Train Epoch: 4 [25600/100000 (26%)]\tLoss: 3.647930\n",
"Tue Oct 30 23:26:38 2018 Train Epoch: 4 [51200/100000 (51%)]\tLoss: 3.421569\n",
"Tue Oct 30 23:28:02 2018 Train Epoch: 4 [76800/100000 (77%)]\tLoss: 3.471924\n",
"\n",
"Test set: Average loss: 3.4881, Accuracy: 2269/10000 (23%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:29:40 2018 Train Epoch: 5 [0/100000 (0%)]\tLoss: 3.263522\n",
"Tue Oct 30 23:31:04 2018 Train Epoch: 5 [25600/100000 (26%)]\tLoss: 3.184380\n",
"Tue Oct 30 23:32:28 2018 Train Epoch: 5 [51200/100000 (51%)]\tLoss: 3.437448\n",
"Tue Oct 30 23:33:52 2018 Train Epoch: 5 [76800/100000 (77%)]\tLoss: 2.971600\n",
"\n",
"Test set: Average loss: 3.3469, Accuracy: 2580/10000 (26%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:35:31 2018 Train Epoch: 6 [0/100000 (0%)]\tLoss: 2.911456\n",
"Tue Oct 30 23:36:54 2018 Train Epoch: 6 [25600/100000 (26%)]\tLoss: 2.977590\n",
"Tue Oct 30 23:38:18 2018 Train Epoch: 6 [51200/100000 (51%)]\tLoss: 2.888722\n",
"Tue Oct 30 23:39:42 2018 Train Epoch: 6 [76800/100000 (77%)]\tLoss: 3.083384\n",
"\n",
"Test set: Average loss: 3.1720, Accuracy: 2861/10000 (29%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:41:21 2018 Train Epoch: 7 [0/100000 (0%)]\tLoss: 2.921553\n",
"Tue Oct 30 23:42:45 2018 Train Epoch: 7 [25600/100000 (26%)]\tLoss: 2.905390\n",
"Tue Oct 30 23:44:08 2018 Train Epoch: 7 [51200/100000 (51%)]\tLoss: 2.958408\n",
"Tue Oct 30 23:45:32 2018 Train Epoch: 7 [76800/100000 (77%)]\tLoss: 2.599775\n",
"\n",
"Test set: Average loss: 3.0303, Accuracy: 3185/10000 (32%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:47:11 2018 Train Epoch: 8 [0/100000 (0%)]\tLoss: 2.647141\n",
"Tue Oct 30 23:48:35 2018 Train Epoch: 8 [25600/100000 (26%)]\tLoss: 2.548127\n",
"Tue Oct 30 23:49:58 2018 Train Epoch: 8 [51200/100000 (51%)]\tLoss: 2.874490\n",
"Tue Oct 30 23:51:22 2018 Train Epoch: 8 [76800/100000 (77%)]\tLoss: 2.496460\n",
"\n",
"Test set: Average loss: 2.9749, Accuracy: 3340/10000 (33%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:53:01 2018 Train Epoch: 9 [0/100000 (0%)]\tLoss: 2.464377\n",
"Tue Oct 30 23:54:25 2018 Train Epoch: 9 [25600/100000 (26%)]\tLoss: 2.417069\n",
"Tue Oct 30 23:55:49 2018 Train Epoch: 9 [51200/100000 (51%)]\tLoss: 2.488547\n",
"Tue Oct 30 23:57:12 2018 Train Epoch: 9 [76800/100000 (77%)]\tLoss: 2.103152\n",
"\n",
"Test set: Average loss: 3.0223, Accuracy: 3240/10000 (32%)\n",
"\n",
"Old model was better new: 32.4old: 33.4\n",
"Tue Oct 30 23:58:51 2018 Train Epoch: 10 [0/100000 (0%)]\tLoss: 2.238087\n",
"Wed Oct 31 00:00:15 2018 Train Epoch: 10 [25600/100000 (26%)]\tLoss: 2.466787\n",
"Wed Oct 31 00:01:39 2018 Train Epoch: 10 [51200/100000 (51%)]\tLoss: 2.474845\n",
"Wed Oct 31 00:03:02 2018 Train Epoch: 10 [76800/100000 (77%)]\tLoss: 2.465969\n",
"\n",
"Test set: Average loss: 2.9739, Accuracy: 3461/10000 (35%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/010.pt\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe11bb69cc0>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe119456160>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"metadata": {
"id": "w9SP7tMAMalF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "6369f07b-212b-46d8-bbbc-1c2c7af227c2"
},
"cell_type": "code",
"source": [
"os.listdir('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/000.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/005.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/007.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/004.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/010.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt')"
],
"execution_count": 52,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[]"
]
},
"metadata": {
"tags": []
},
"execution_count": 52
}
]
},
{
"metadata": {
"id": "4FGXjLjtFbHa",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 6: ImageNet!\n",
"ImageNet is the big granddaddy of supervised vision datasets. It's the dataset that got the whole deep learning train running. We thought it would be fun for you guys to try your hand at it.\n",
"\n",
"A few notes:\n",
"- Downloading ImageNet should take about 7 minutues. Extracting should take another 7 minutes. So run this code, then go have a coffee or whatever.\n",
"- The data will be removed after 24 hours or so. That's just how Colab works. Each time you reset the environment, you will also have to redownload ImageNet. \n",
" - I would recommend against storing it on your on Google Drive though as that is way slower to read during training.\n",
" - Through extensive testing, we have found that the fastest way to get the data is the code we provide (faster than saving to google drive, uploading by hand, etc.) \n",
"- Reading images directly with PIL seems to have issues on Colab. Instead try cv2.imread.\n",
" - cv2.imread returns images in bgr order. To go from bgr -> rgb, use `im = im[:, :, ::-1]`\n",
"- The images are provided such that the minimum side length is at least 128 unless the original image was too small for it to be shrunk.\n",
" - transforms.Resize will be your friend\n",
"- It seams you cannot use a batch size > 64 x 128 x 128 x 3 while still using multiprocessing due to Colab limitations. You can use larger batches if you set num_workers = 0, but it will be much slower.\n",
"- This dataset has 1000 classes, whereas Tiny ImageNet only had 200.\n",
"- You will probably want to save your results more than once every epoch since Colab may spurriously shut down your session.\n",
"- We know this takes a long time to run. Try leaving it overnight. We don't expect you to get state-of-the-art performance.\n",
"- `glob.glob` is a good Python function for reading lots of folder/file names, but it does not return a sorted list.\n",
"\n",
"\n"
]
},
{
"metadata": {
"id": "68XKUg9hF5Io",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 187
},
"outputId": "d70ccd0a-4d3f-4fa1-fff8-496a899669ff"
},
"cell_type": "code",
"source": [
"# Downloads and extracts the data\n",
"if not os.path.exists('imagenet128'):\n",
" !echo $(date +%x_%r)\n",
" !gsutil cp gs://imagenet-cropped/imagenet128.tar .\n",
" !echo $(date +%x_%r)\n",
" !tar -xf imagenet128.tar\n",
" !echo $(date +%x_%r)\n",
" !ls imagenet128/train | head -4"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"10/31/2018_08:39:48 PM\n",
"Copying gs://imagenet-cropped/imagenet128.tar...\n",
"\\ [1 files][ 23.7 GiB/ 23.7 GiB] 55.7 MiB/s \n",
"Operation completed over 1 objects/23.7 GiB. \n",
"10/31/2018_08:47:28 PM\n",
"10/31/2018_08:53:24 PM\n",
"n01440764\n",
"n01443537\n",
"n01484850\n",
"n01491361\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "-DqAvGsk89qt",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
" # Sets up synset data. Have a look at what is in these dictionaries as they may help you with debugging.\n",
"import json\n",
"imagenet_synset_data = json.load(open('imagenet128/imagenet_synsets.json'))\n",
"synset_id_to_cls = {val['id']: int(key) for key, val in imagenet_synset_data.items()}"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "4zD1uP7lW5ue",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"!export CUDA_LAUNCH_BLOCKING=0"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "0bUNNoLlGWgc",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"# Data loader\n",
"import cv2\n",
"class ImageFolderLoader(torch.utils.data.Dataset):\n",
" def __init__(self, folder, transform=None):\n",
" self.folder = folder\n",
" self.transform = transform\n",
" mydict = {}\n",
"\n",
" for directory in os.listdir(self.folder):\n",
" for file in os.listdir(self.folder + '/' + directory):\n",
" filename =self.folder + '/' + directory + '/' + file\n",
" mydict[filename] = synset_id_to_cls[directory]\n",
" \n",
" self.dictlist = []\n",
" for key, value in mydict.items():\n",
" temp = [key, value]\n",
" self.dictlist.append(temp)\n",
" del mydict\n",
" \n",
"\n",
" def __len__(self):\n",
" return len(self.dictlist)\n",
" def __getitem__(self, idx):\n",
" #dictlist is a list of tuples ('filename', classNumber)\n",
" filename = self.dictlist[idx][0]\n",
" label = self.dictlist[idx][1]\n",
" #cv2.imread returns images in bgr order. To go from bgr -> rgb, use im = im[:, :, ::-1]\n",
" im = cv2.imread(filename)\n",
" im = im[:, :, ::-1]\n",
" imtensor = torch.FloatTensor(im.copy()).permute((2, 0, 1))\n",
" #print('Pre transform' + str(imtensor.shape))\n",
" if self.transform:\n",
" imtensor = self.transform(imtensor)\n",
" #print('Post transform' + str(imtensor.shape))\n",
" \n",
" labeltensor = torch.LongTensor([label])\n",
" #print('Labeltensor' + str(labeltensor.shape))\n",
" return (imtensor, labeltensor)\n"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "IwBV9dcTGYqx",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"train_transforms = transforms.Compose([transforms.ToPILImage(), transforms.Resize((64,64)), transforms.ToTensor()])\n",
"test_transforms = transforms.Compose([transforms.ToPILImage(), transforms.Resize((64,64)), transforms.ToTensor()])\n",
"\n",
"data_train = ImageFolderLoader('imagenet128/train', transform=train_transforms)\n",
"assert(len(data_train) == 1281167)\n",
"data_test = ImageFolderLoader('imagenet128/val', transform=test_transforms)\n",
"assert(len(data_test) == 50000)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "hp9pprc_-GEf",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"data_train[0]"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "ZAwdjaIuGj34",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"class FullImagenetNet(TinyImagenetNet):\n",
" def __init__(self):\n",
" super(FullImagenetNet, self).__init__()\n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # now just using a sample of darknet\n",
" self.conv4 = nn.Conv2d(3, 128, kernel_size=3, stride=1)\n",
" self.conv5 = nn.Conv2d(128, 64, kernel_size=1, stride=1)\n",
" self.conv6 = nn.Conv2d(64, 128, kernel_size=3, stride=1)\n",
" self.pool7 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv8 = nn.Conv2d(128, 256, kernel_size=3, stride = 1)\n",
" self.conv9 = nn.Conv2d(256, 128, kernel_size=1, stride=1)\n",
" self.conv10 = nn.Conv2d(128, 256, kernel_size=3, stride=1)\n",
" self.pool11 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv16 = nn.Conv2d(256, 512, kernel_size=3, stride=1)\n",
" self.pool17 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.avg4b4 = nn.AvgPool2d(kernel_size=5, stride=1, padding=0)\n",
" self.fc = nn.Linear(512, 1000)\n",
" \n",
" \n",
" #use for save_best_model\n",
" self.best = 0.0\n",
" \n",
" def forward(self, x):# TODO define the forward pass\n",
" #raise NotImplementedError('Need to define the forward pass')\n",
" # dimms with batch size 10\n",
" x = x # ([64, 3, 64, 64])\n",
" x = F.relu(self.conv4(x)) # [10, 128, 62, 62]\n",
" x = F.relu(self.conv5(x)) # [10, 64, 62, 62]\n",
" x = F.relu(self.conv6(x)) # [10, 128, 60, 60]\n",
" x = self.pool7(x) # [10, 128, 30, 30]\n",
" x = F.relu(self.conv8(x)) # [10, 256, 28, 28]\n",
" x = F.relu(self.conv9(x)) #[10, 128, 28, 28]\n",
" x = F.relu(self.conv10(x)) # [10, 256, 26, 26]\n",
" x = self.pool11(x) # [10, 256, 13, 13]\n",
" x = F.relu(self.conv16(x)) # [10, 512, 11, 11]\n",
" x = self.pool17(x) # [10, 512, 5, 5]\n",
" x = self.avg4b4(x) # [10, 512, 1, 1]\n",
" x = x.view(-1, 512)\n",
" x = self.fc(x)\n",
" \n",
" \n",
" return F.log_softmax(x, dim=1)\n"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "tL0yjhP_Gnd0",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 3978
},
"outputId": "797280ec-2e96-4880-d310-b2ad9d790fce"
},
"cell_type": "code",
"source": [
"BATCH_SIZE = 64\n",
"TEST_BATCH_SIZE = 50\n",
"EPOCHS = 200\n",
"LEARNING_RATE = 0.1#0.01\n",
"MOMENTUM = 0.9\n",
"USE_CUDA = True\n",
"PRINT_INTERVAL = 100\n",
"WEIGHT_DECAY = 0.0005\n",
"CHECKPOINT_PATH = BASE_PATH + 'imagenet_full/checkpoints'\n",
"LOG_PATH = BASE_PATH + 'imagenet_full/' + 'log.pkl'\n",
"\n",
"use_cuda = USE_CUDA and torch.cuda.is_available()\n",
"\n",
"device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
"print('Using device', device)\n",
"import multiprocessing\n",
"print('num cpus:', multiprocessing.cpu_count())\n",
"\n",
"kwargs = {'num_workers': multiprocessing.cpu_count(),\n",
" 'pin_memory': True} if use_cuda else {}\n",
"\n",
"\n",
"class_names = sorted([(int(key), val['label'].split(',')[0]) for key, val in imagenet_synset_data.items()])\n",
"name_to_class = {line[1]: line[0] for line in class_names}\n",
"class_names = [line[1] for line in class_names]\n",
"#print(data_train)\n",
"train_loader = torch.utils.data.DataLoader(data_train, batch_size=BATCH_SIZE,\n",
" shuffle=True, **kwargs)\n",
"test_loader = torch.utils.data.DataLoader(data_test, batch_size=TEST_BATCH_SIZE,\n",
" shuffle=False, **kwargs)\n",
"\n",
"model = FullImagenetNet().to(device)\n",
"optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n",
"start_epoch = model.load_last_model(CHECKPOINT_PATH)\n",
"\n",
"\n",
"try:\n",
" for epoch in range(start_epoch, EPOCHS + 1):\n",
" train(model, device, train_loader, optimizer, epoch, PRINT_INTERVAL)\n",
" correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy =test(model, device, test_loader, False)\n",
" model.save_best_model(test_accuracy, CHECKPOINT_PATH + '/%03d.pt' % epoch)\n",
"\n",
"\n",
"except KeyboardInterrupt as ke:\n",
" print('Interrupted')\n",
"except:\n",
" import traceback\n",
" traceback.print_exc()\n",
"finally:\n",
" model.save_model(CHECKPOINT_PATH + '/%03d.pt' % epoch, 0)"
],
"execution_count": 29,
"outputs": [
{
"output_type": "stream",
"text": [
"Using device cuda\n",
"num cpus: 2\n",
"Restoring:\n",
"conv4.weight -> \ttorch.Size([128, 3, 3, 3]) = 0MB\n",
"conv4.bias -> \ttorch.Size([128]) = 0MB\n",
"conv5.weight -> \ttorch.Size([64, 128, 1, 1]) = 0MB\n",
"conv5.bias -> \ttorch.Size([64]) = 0MB\n",
"conv6.weight -> \ttorch.Size([128, 64, 3, 3]) = 0MB\n",
"conv6.bias -> \ttorch.Size([128]) = 0MB\n",
"conv8.weight -> \ttorch.Size([256, 128, 3, 3]) = 1MB\n",
"conv8.bias -> \ttorch.Size([256]) = 0MB\n",
"conv9.weight -> \ttorch.Size([128, 256, 1, 1]) = 0MB\n",
"conv9.bias -> \ttorch.Size([128]) = 0MB\n",
"conv10.weight -> \ttorch.Size([256, 128, 3, 3]) = 1MB\n",
"conv10.bias -> \ttorch.Size([256]) = 0MB\n",
"conv16.weight -> \ttorch.Size([512, 256, 3, 3]) = 4MB\n",
"conv16.bias -> \ttorch.Size([512]) = 0MB\n",
"fc.weight -> \ttorch.Size([1000, 512]) = 2MB\n",
"fc.bias -> \ttorch.Size([1000]) = 0MB\n",
"\n",
"Restored all variables\n",
"No new variables\n",
"Restored /gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt\n",
"Thu Nov 1 03:13:50 2018 Train Epoch: 0 [0/1281167 (0%)]\tLoss: 6.905530\n",
"Thu Nov 1 03:14:15 2018 Train Epoch: 0 [6400/1281167 (0%)]\tLoss: 6.915213\n",
"Thu Nov 1 03:14:39 2018 Train Epoch: 0 [12800/1281167 (1%)]\tLoss: 6.906976\n",
"Thu Nov 1 03:15:02 2018 Train Epoch: 0 [19200/1281167 (1%)]\tLoss: 6.903210\n",
"Thu Nov 1 03:15:26 2018 Train Epoch: 0 [25600/1281167 (2%)]\tLoss: 6.894549\n",
"Thu Nov 1 03:15:50 2018 Train Epoch: 0 [32000/1281167 (2%)]\tLoss: 6.913801\n",
"Thu Nov 1 03:16:13 2018 Train Epoch: 0 [38400/1281167 (3%)]\tLoss: 6.911386\n",
"Thu Nov 1 03:16:37 2018 Train Epoch: 0 [44800/1281167 (3%)]\tLoss: 6.903984\n",
"Thu Nov 1 03:17:00 2018 Train Epoch: 0 [51200/1281167 (4%)]\tLoss: 6.913480\n",
"Thu Nov 1 03:17:23 2018 Train Epoch: 0 [57600/1281167 (4%)]\tLoss: 6.921412\n",
"Thu Nov 1 03:17:47 2018 Train Epoch: 0 [64000/1281167 (5%)]\tLoss: 6.917922\n",
"Thu Nov 1 03:18:10 2018 Train Epoch: 0 [70400/1281167 (5%)]\tLoss: 6.917318\n",
"Thu Nov 1 03:18:34 2018 Train Epoch: 0 [76800/1281167 (6%)]\tLoss: 6.906056\n",
"Thu Nov 1 03:18:57 2018 Train Epoch: 0 [83200/1281167 (6%)]\tLoss: 6.903532\n",
"Thu Nov 1 03:19:21 2018 Train Epoch: 0 [89600/1281167 (7%)]\tLoss: 6.920099\n",
"Thu Nov 1 03:19:45 2018 Train Epoch: 0 [96000/1281167 (7%)]\tLoss: 6.896837\n",
"Thu Nov 1 03:20:08 2018 Train Epoch: 0 [102400/1281167 (8%)]\tLoss: 6.901331\n",
"Thu Nov 1 03:20:31 2018 Train Epoch: 0 [108800/1281167 (8%)]\tLoss: 6.912484\n",
"Thu Nov 1 03:20:54 2018 Train Epoch: 0 [115200/1281167 (9%)]\tLoss: 6.920592\n",
"Thu Nov 1 03:21:17 2018 Train Epoch: 0 [121600/1281167 (9%)]\tLoss: 6.903300\n",
"Thu Nov 1 03:21:41 2018 Train Epoch: 0 [128000/1281167 (10%)]\tLoss: 6.907892\n",
"Thu Nov 1 03:22:04 2018 Train Epoch: 0 [134400/1281167 (10%)]\tLoss: 6.911784\n",
"Thu Nov 1 03:22:27 2018 Train Epoch: 0 [140800/1281167 (11%)]\tLoss: 6.901532\n",
"Thu Nov 1 03:22:50 2018 Train Epoch: 0 [147200/1281167 (11%)]\tLoss: 6.914480\n",
"Thu Nov 1 03:23:14 2018 Train Epoch: 0 [153600/1281167 (12%)]\tLoss: 6.903480\n",
"Thu Nov 1 03:23:38 2018 Train Epoch: 0 [160000/1281167 (12%)]\tLoss: 6.903686\n",
"Thu Nov 1 03:24:01 2018 Train Epoch: 0 [166400/1281167 (13%)]\tLoss: 6.924271\n",
"Thu Nov 1 03:24:25 2018 Train Epoch: 0 [172800/1281167 (13%)]\tLoss: 6.912843\n",
"Thu Nov 1 03:24:50 2018 Train Epoch: 0 [179200/1281167 (14%)]\tLoss: 6.898242\n",
"Thu Nov 1 03:25:13 2018 Train Epoch: 0 [185600/1281167 (14%)]\tLoss: 6.926912\n",
"Thu Nov 1 03:25:36 2018 Train Epoch: 0 [192000/1281167 (15%)]\tLoss: 6.918694\n",
"Thu Nov 1 03:25:59 2018 Train Epoch: 0 [198400/1281167 (15%)]\tLoss: 6.920316\n",
"Thu Nov 1 03:26:22 2018 Train Epoch: 0 [204800/1281167 (16%)]\tLoss: 6.907867\n",
"Thu Nov 1 03:26:45 2018 Train Epoch: 0 [211200/1281167 (16%)]\tLoss: 6.905494\n",
"Thu Nov 1 03:27:08 2018 Train Epoch: 0 [217600/1281167 (17%)]\tLoss: 6.919506\n",
"Thu Nov 1 03:27:31 2018 Train Epoch: 0 [224000/1281167 (17%)]\tLoss: 6.919896\n",
"Thu Nov 1 03:27:55 2018 Train Epoch: 0 [230400/1281167 (18%)]\tLoss: 6.913472\n",
"Thu Nov 1 03:28:17 2018 Train Epoch: 0 [236800/1281167 (18%)]\tLoss: 6.911576\n",
"Thu Nov 1 03:28:40 2018 Train Epoch: 0 [243200/1281167 (19%)]\tLoss: 6.910405\n",
"Thu Nov 1 03:29:04 2018 Train Epoch: 0 [249600/1281167 (19%)]\tLoss: 6.914724\n",
"Thu Nov 1 03:29:27 2018 Train Epoch: 0 [256000/1281167 (20%)]\tLoss: 6.914860\n",
"Thu Nov 1 03:29:50 2018 Train Epoch: 0 [262400/1281167 (20%)]\tLoss: 6.917733\n",
"Thu Nov 1 03:30:13 2018 Train Epoch: 0 [268800/1281167 (21%)]\tLoss: 6.916535\n",
"Thu Nov 1 03:30:37 2018 Train Epoch: 0 [275200/1281167 (21%)]\tLoss: 6.895416\n",
"Thu Nov 1 03:31:00 2018 Train Epoch: 0 [281600/1281167 (22%)]\tLoss: 6.915490\n",
"Thu Nov 1 03:31:24 2018 Train Epoch: 0 [288000/1281167 (22%)]\tLoss: 6.927084\n",
"Thu Nov 1 03:31:48 2018 Train Epoch: 0 [294400/1281167 (23%)]\tLoss: 6.900946\n",
"Thu Nov 1 03:32:12 2018 Train Epoch: 0 [300800/1281167 (23%)]\tLoss: 6.930200\n",
"Thu Nov 1 03:32:35 2018 Train Epoch: 0 [307200/1281167 (24%)]\tLoss: 6.912046\n",
"Thu Nov 1 03:32:58 2018 Train Epoch: 0 [313600/1281167 (24%)]\tLoss: 6.917764\n",
"Thu Nov 1 03:33:22 2018 Train Epoch: 0 [320000/1281167 (25%)]\tLoss: 6.892279\n",
"Thu Nov 1 03:33:47 2018 Train Epoch: 0 [326400/1281167 (25%)]\tLoss: 6.924290\n",
"Thu Nov 1 03:34:11 2018 Train Epoch: 0 [332800/1281167 (26%)]\tLoss: 6.902865\n",
"Thu Nov 1 03:34:34 2018 Train Epoch: 0 [339200/1281167 (26%)]\tLoss: 6.913850\n",
"Thu Nov 1 03:34:57 2018 Train Epoch: 0 [345600/1281167 (27%)]\tLoss: 6.907212\n",
"Thu Nov 1 03:35:21 2018 Train Epoch: 0 [352000/1281167 (27%)]\tLoss: 6.916387\n",
"Thu Nov 1 03:35:44 2018 Train Epoch: 0 [358400/1281167 (28%)]\tLoss: 6.928115\n",
"Thu Nov 1 03:36:06 2018 Train Epoch: 0 [364800/1281167 (28%)]\tLoss: 6.894051\n",
"Thu Nov 1 03:36:29 2018 Train Epoch: 0 [371200/1281167 (29%)]\tLoss: 6.889081\n",
"Thu Nov 1 03:36:52 2018 Train Epoch: 0 [377600/1281167 (29%)]\tLoss: 6.902927\n",
"Thu Nov 1 03:37:17 2018 Train Epoch: 0 [384000/1281167 (30%)]\tLoss: 6.913784\n",
"Thu Nov 1 03:37:40 2018 Train Epoch: 0 [390400/1281167 (30%)]\tLoss: 6.917893\n",
"Thu Nov 1 03:38:03 2018 Train Epoch: 0 [396800/1281167 (31%)]\tLoss: 6.917501\n",
"Thu Nov 1 03:38:26 2018 Train Epoch: 0 [403200/1281167 (31%)]\tLoss: 6.906955\n",
"Thu Nov 1 03:38:49 2018 Train Epoch: 0 [409600/1281167 (32%)]\tLoss: 6.915683\n",
"Thu Nov 1 03:39:12 2018 Train Epoch: 0 [416000/1281167 (32%)]\tLoss: 6.906111\n",
"Thu Nov 1 03:39:35 2018 Train Epoch: 0 [422400/1281167 (33%)]\tLoss: 6.911583\n",
"Thu Nov 1 03:39:59 2018 Train Epoch: 0 [428800/1281167 (33%)]\tLoss: 6.915631\n",
"Thu Nov 1 03:40:22 2018 Train Epoch: 0 [435200/1281167 (34%)]\tLoss: 6.881934\n",
"Thu Nov 1 03:40:46 2018 Train Epoch: 0 [441600/1281167 (34%)]\tLoss: 6.905287\n",
"Thu Nov 1 03:41:09 2018 Train Epoch: 0 [448000/1281167 (35%)]\tLoss: 6.906995\n",
"Thu Nov 1 03:41:33 2018 Train Epoch: 0 [454400/1281167 (35%)]\tLoss: 6.905661\n",
"Thu Nov 1 03:41:56 2018 Train Epoch: 0 [460800/1281167 (36%)]\tLoss: 6.927936\n",
"Thu Nov 1 03:42:22 2018 Train Epoch: 0 [467200/1281167 (36%)]\tLoss: 6.908838\n",
"Thu Nov 1 03:42:48 2018 Train Epoch: 0 [473600/1281167 (37%)]\tLoss: 6.901138\n",
"Thu Nov 1 03:43:12 2018 Train Epoch: 0 [480000/1281167 (37%)]\tLoss: 6.915953\n",
"Thu Nov 1 03:43:35 2018 Train Epoch: 0 [486400/1281167 (38%)]\tLoss: 6.907689\n",
"Thu Nov 1 03:43:59 2018 Train Epoch: 0 [492800/1281167 (38%)]\tLoss: 6.914100\n",
"Thu Nov 1 03:44:22 2018 Train Epoch: 0 [499200/1281167 (39%)]\tLoss: 6.915280\n",
"Thu Nov 1 03:44:45 2018 Train Epoch: 0 [505600/1281167 (39%)]\tLoss: 6.920105\n",
"Thu Nov 1 03:45:09 2018 Train Epoch: 0 [512000/1281167 (40%)]\tLoss: 6.903097\n",
"Thu Nov 1 03:45:32 2018 Train Epoch: 0 [518400/1281167 (40%)]\tLoss: 6.907869\n",
"Thu Nov 1 03:45:55 2018 Train Epoch: 0 [524800/1281167 (41%)]\tLoss: 6.905593\n",
"Thu Nov 1 03:46:18 2018 Train Epoch: 0 [531200/1281167 (41%)]\tLoss: 6.909816\n",
"Thu Nov 1 03:46:42 2018 Train Epoch: 0 [537600/1281167 (42%)]\tLoss: 6.896438\n",
"Thu Nov 1 03:47:05 2018 Train Epoch: 0 [544000/1281167 (42%)]\tLoss: 6.907209\n",
"Thu Nov 1 03:47:28 2018 Train Epoch: 0 [550400/1281167 (43%)]\tLoss: 6.908332\n",
"Thu Nov 1 03:47:52 2018 Train Epoch: 0 [556800/1281167 (43%)]\tLoss: 6.906932\n",
"Thu Nov 1 03:48:14 2018 Train Epoch: 0 [563200/1281167 (44%)]\tLoss: 6.910619\n",
"Thu Nov 1 03:48:38 2018 Train Epoch: 0 [569600/1281167 (44%)]\tLoss: 6.903950\n",
"Thu Nov 1 03:49:01 2018 Train Epoch: 0 [576000/1281167 (45%)]\tLoss: 6.917201\n",
"Thu Nov 1 03:49:24 2018 Train Epoch: 0 [582400/1281167 (45%)]\tLoss: 6.899111\n",
"Thu Nov 1 03:49:46 2018 Train Epoch: 0 [588800/1281167 (46%)]\tLoss: 6.891315\n",
"Thu Nov 1 03:50:10 2018 Train Epoch: 0 [595200/1281167 (46%)]\tLoss: 6.913540\n",
"Thu Nov 1 03:50:33 2018 Train Epoch: 0 [601600/1281167 (47%)]\tLoss: 6.918234\n",
"Thu Nov 1 03:50:56 2018 Train Epoch: 0 [608000/1281167 (47%)]\tLoss: 6.923501\n",
"Thu Nov 1 03:51:18 2018 Train Epoch: 0 [614400/1281167 (48%)]\tLoss: 6.924489\n",
"Thu Nov 1 03:51:41 2018 Train Epoch: 0 [620800/1281167 (48%)]\tLoss: 6.910414\n",
"Thu Nov 1 03:52:04 2018 Train Epoch: 0 [627200/1281167 (49%)]\tLoss: 6.900335\n",
"Thu Nov 1 03:52:27 2018 Train Epoch: 0 [633600/1281167 (49%)]\tLoss: 6.915105\n",
"Thu Nov 1 03:52:51 2018 Train Epoch: 0 [640000/1281167 (50%)]\tLoss: 6.904331\n",
"Thu Nov 1 03:53:14 2018 Train Epoch: 0 [646400/1281167 (50%)]\tLoss: 6.920506\n",
"Thu Nov 1 03:53:37 2018 Train Epoch: 0 [652800/1281167 (51%)]\tLoss: 6.912586\n",
"Thu Nov 1 03:54:00 2018 Train Epoch: 0 [659200/1281167 (51%)]\tLoss: 6.905747\n",
"Thu Nov 1 03:54:23 2018 Train Epoch: 0 [665600/1281167 (52%)]\tLoss: 6.919553\n",
"Thu Nov 1 03:54:46 2018 Train Epoch: 0 [672000/1281167 (52%)]\tLoss: 6.901967\n",
"Thu Nov 1 03:55:09 2018 Train Epoch: 0 [678400/1281167 (53%)]\tLoss: 6.899582\n",
"Thu Nov 1 03:55:32 2018 Train Epoch: 0 [684800/1281167 (53%)]\tLoss: 6.907303\n",
"Thu Nov 1 03:55:55 2018 Train Epoch: 0 [691200/1281167 (54%)]\tLoss: 6.910735\n",
"Thu Nov 1 03:56:18 2018 Train Epoch: 0 [697600/1281167 (54%)]\tLoss: 6.902495\n",
"Thu Nov 1 03:56:40 2018 Train Epoch: 0 [704000/1281167 (55%)]\tLoss: 6.907861\n",
"Thu Nov 1 03:57:03 2018 Train Epoch: 0 [710400/1281167 (55%)]\tLoss: 6.905015\n",
"Thu Nov 1 03:57:26 2018 Train Epoch: 0 [716800/1281167 (56%)]\tLoss: 6.908472\n",
"Thu Nov 1 03:57:50 2018 Train Epoch: 0 [723200/1281167 (56%)]\tLoss: 6.898434\n",
"Thu Nov 1 03:58:14 2018 Train Epoch: 0 [729600/1281167 (57%)]\tLoss: 6.901069\n",
"Thu Nov 1 03:58:38 2018 Train Epoch: 0 [736000/1281167 (57%)]\tLoss: 6.892691\n",
"Thu Nov 1 03:59:03 2018 Train Epoch: 0 [742400/1281167 (58%)]\tLoss: 6.921463\n",
"Thu Nov 1 03:59:26 2018 Train Epoch: 0 [748800/1281167 (58%)]\tLoss: 6.902192\n",
"Thu Nov 1 03:59:50 2018 Train Epoch: 0 [755200/1281167 (59%)]\tLoss: 6.899998\n",
"Thu Nov 1 04:00:14 2018 Train Epoch: 0 [761600/1281167 (59%)]\tLoss: 6.895144\n",
"Thu Nov 1 04:00:38 2018 Train Epoch: 0 [768000/1281167 (60%)]\tLoss: 6.902062\n",
"Thu Nov 1 04:01:04 2018 Train Epoch: 0 [774400/1281167 (60%)]\tLoss: 6.901936\n",
"Thu Nov 1 04:01:27 2018 Train Epoch: 0 [780800/1281167 (61%)]\tLoss: 6.903090\n",
"Thu Nov 1 04:01:51 2018 Train Epoch: 0 [787200/1281167 (61%)]\tLoss: 6.904884\n",
"Thu Nov 1 04:02:15 2018 Train Epoch: 0 [793600/1281167 (62%)]\tLoss: 6.923146\n",
"Thu Nov 1 04:02:38 2018 Train Epoch: 0 [800000/1281167 (62%)]\tLoss: 6.913280\n",
"Thu Nov 1 04:03:02 2018 Train Epoch: 0 [806400/1281167 (63%)]\tLoss: 6.906809\n",
"Thu Nov 1 04:03:25 2018 Train Epoch: 0 [812800/1281167 (63%)]\tLoss: 6.913114\n",
"Thu Nov 1 04:03:48 2018 Train Epoch: 0 [819200/1281167 (64%)]\tLoss: 6.915143\n",
"Thu Nov 1 04:04:10 2018 Train Epoch: 0 [825600/1281167 (64%)]\tLoss: 6.921178\n",
"Thu Nov 1 04:04:33 2018 Train Epoch: 0 [832000/1281167 (65%)]\tLoss: 6.901847\n",
"Thu Nov 1 04:04:56 2018 Train Epoch: 0 [838400/1281167 (65%)]\tLoss: 6.938429\n",
"Thu Nov 1 04:05:19 2018 Train Epoch: 0 [844800/1281167 (66%)]\tLoss: 6.895684\n",
"Thu Nov 1 04:05:41 2018 Train Epoch: 0 [851200/1281167 (66%)]\tLoss: 6.901691\n",
"Thu Nov 1 04:06:04 2018 Train Epoch: 0 [857600/1281167 (67%)]\tLoss: 6.911649\n",
"Thu Nov 1 04:06:27 2018 Train Epoch: 0 [864000/1281167 (67%)]\tLoss: 6.923692\n",
"Thu Nov 1 04:06:50 2018 Train Epoch: 0 [870400/1281167 (68%)]\tLoss: 6.900036\n",
"Thu Nov 1 04:07:13 2018 Train Epoch: 0 [876800/1281167 (68%)]\tLoss: 6.927470\n",
"Thu Nov 1 04:07:35 2018 Train Epoch: 0 [883200/1281167 (69%)]\tLoss: 6.910205\n",
"Thu Nov 1 04:07:58 2018 Train Epoch: 0 [889600/1281167 (69%)]\tLoss: 6.874023\n",
"Thu Nov 1 04:08:21 2018 Train Epoch: 0 [896000/1281167 (70%)]\tLoss: 6.903159\n",
"Thu Nov 1 04:08:44 2018 Train Epoch: 0 [902400/1281167 (70%)]\tLoss: 6.908606\n",
"Thu Nov 1 04:09:07 2018 Train Epoch: 0 [908800/1281167 (71%)]\tLoss: 6.887091\n",
"Thu Nov 1 04:09:30 2018 Train Epoch: 0 [915200/1281167 (71%)]\tLoss: 6.902290\n",
"Thu Nov 1 04:09:53 2018 Train Epoch: 0 [921600/1281167 (72%)]\tLoss: 6.919454\n",
"Thu Nov 1 04:10:15 2018 Train Epoch: 0 [928000/1281167 (72%)]\tLoss: 6.910271\n",
"Thu Nov 1 04:10:38 2018 Train Epoch: 0 [934400/1281167 (73%)]\tLoss: 6.900114\n",
"Thu Nov 1 04:11:01 2018 Train Epoch: 0 [940800/1281167 (73%)]\tLoss: 6.925539\n",
"Thu Nov 1 04:11:24 2018 Train Epoch: 0 [947200/1281167 (74%)]\tLoss: 6.923165\n",
"Thu Nov 1 04:11:47 2018 Train Epoch: 0 [953600/1281167 (74%)]\tLoss: 6.901720\n",
"Thu Nov 1 04:12:10 2018 Train Epoch: 0 [960000/1281167 (75%)]\tLoss: 6.907963\n",
"Thu Nov 1 04:12:32 2018 Train Epoch: 0 [966400/1281167 (75%)]\tLoss: 6.921975\n",
"Thu Nov 1 04:12:54 2018 Train Epoch: 0 [972800/1281167 (76%)]\tLoss: 6.915666\n",
"Thu Nov 1 04:13:17 2018 Train Epoch: 0 [979200/1281167 (76%)]\tLoss: 6.914638\n",
"Thu Nov 1 04:13:40 2018 Train Epoch: 0 [985600/1281167 (77%)]\tLoss: 6.900302\n",
"Thu Nov 1 04:14:02 2018 Train Epoch: 0 [992000/1281167 (77%)]\tLoss: 6.907050\n",
"Thu Nov 1 04:14:25 2018 Train Epoch: 0 [998400/1281167 (78%)]\tLoss: 6.919683\n",
"Thu Nov 1 04:14:48 2018 Train Epoch: 0 [1004800/1281167 (78%)]\tLoss: 6.910315\n",
"Thu Nov 1 04:15:10 2018 Train Epoch: 0 [1011200/1281167 (79%)]\tLoss: 6.920113\n",
"Thu Nov 1 04:15:33 2018 Train Epoch: 0 [1017600/1281167 (79%)]\tLoss: 6.906448\n",
"Thu Nov 1 04:15:56 2018 Train Epoch: 0 [1024000/1281167 (80%)]\tLoss: 6.919313\n",
"Thu Nov 1 04:16:18 2018 Train Epoch: 0 [1030400/1281167 (80%)]\tLoss: 6.911633\n",
"Thu Nov 1 04:16:40 2018 Train Epoch: 0 [1036800/1281167 (81%)]\tLoss: 6.914162\n",
"Thu Nov 1 04:17:03 2018 Train Epoch: 0 [1043200/1281167 (81%)]\tLoss: 6.921151\n",
"Thu Nov 1 04:17:25 2018 Train Epoch: 0 [1049600/1281167 (82%)]\tLoss: 6.910539\n",
"Thu Nov 1 04:17:48 2018 Train Epoch: 0 [1056000/1281167 (82%)]\tLoss: 6.919050\n",
"Thu Nov 1 04:18:10 2018 Train Epoch: 0 [1062400/1281167 (83%)]\tLoss: 6.908661\n",
"Thu Nov 1 04:18:33 2018 Train Epoch: 0 [1068800/1281167 (83%)]\tLoss: 6.904510\n",
"Thu Nov 1 04:18:55 2018 Train Epoch: 0 [1075200/1281167 (84%)]\tLoss: 6.905267\n",
"Thu Nov 1 04:19:18 2018 Train Epoch: 0 [1081600/1281167 (84%)]\tLoss: 6.894763\n",
"Thu Nov 1 04:19:41 2018 Train Epoch: 0 [1088000/1281167 (85%)]\tLoss: 6.916194\n",
"Thu Nov 1 04:20:03 2018 Train Epoch: 0 [1094400/1281167 (85%)]\tLoss: 6.909140\n",
"Thu Nov 1 04:20:25 2018 Train Epoch: 0 [1100800/1281167 (86%)]\tLoss: 6.924272\n",
"Thu Nov 1 04:20:48 2018 Train Epoch: 0 [1107200/1281167 (86%)]\tLoss: 6.893477\n",
"Thu Nov 1 04:21:11 2018 Train Epoch: 0 [1113600/1281167 (87%)]\tLoss: 6.909297\n",
"Thu Nov 1 04:21:34 2018 Train Epoch: 0 [1120000/1281167 (87%)]\tLoss: 6.903625\n",
"Thu Nov 1 04:21:56 2018 Train Epoch: 0 [1126400/1281167 (88%)]\tLoss: 6.898579\n",
"Thu Nov 1 04:22:19 2018 Train Epoch: 0 [1132800/1281167 (88%)]\tLoss: 6.906099\n",
"Thu Nov 1 04:22:41 2018 Train Epoch: 0 [1139200/1281167 (89%)]\tLoss: 6.911266\n",
"Thu Nov 1 04:23:04 2018 Train Epoch: 0 [1145600/1281167 (89%)]\tLoss: 6.913440\n",
"Thu Nov 1 04:23:27 2018 Train Epoch: 0 [1152000/1281167 (90%)]\tLoss: 6.904614\n",
"Thu Nov 1 04:23:50 2018 Train Epoch: 0 [1158400/1281167 (90%)]\tLoss: 6.912544\n",
"Thu Nov 1 04:24:12 2018 Train Epoch: 0 [1164800/1281167 (91%)]\tLoss: 6.887463\n",
"Thu Nov 1 04:24:35 2018 Train Epoch: 0 [1171200/1281167 (91%)]\tLoss: 6.907408\n",
"Thu Nov 1 04:24:57 2018 Train Epoch: 0 [1177600/1281167 (92%)]\tLoss: 6.901124\n",
"Thu Nov 1 04:25:20 2018 Train Epoch: 0 [1184000/1281167 (92%)]\tLoss: 6.918556\n",
"Thu Nov 1 04:25:43 2018 Train Epoch: 0 [1190400/1281167 (93%)]\tLoss: 6.919122\n",
"Thu Nov 1 04:26:05 2018 Train Epoch: 0 [1196800/1281167 (93%)]\tLoss: 6.904559\n",
"Thu Nov 1 04:26:28 2018 Train Epoch: 0 [1203200/1281167 (94%)]\tLoss: 6.905938\n",
"Thu Nov 1 04:26:50 2018 Train Epoch: 0 [1209600/1281167 (94%)]\tLoss: 6.915270\n",
"Thu Nov 1 04:27:12 2018 Train Epoch: 0 [1216000/1281167 (95%)]\tLoss: 6.896585\n",
"Thu Nov 1 04:27:35 2018 Train Epoch: 0 [1222400/1281167 (95%)]\tLoss: 6.910329\n",
"Thu Nov 1 04:27:57 2018 Train Epoch: 0 [1228800/1281167 (96%)]\tLoss: 6.909390\n",
"Thu Nov 1 04:28:19 2018 Train Epoch: 0 [1235200/1281167 (96%)]\tLoss: 6.909443\n",
"Thu Nov 1 04:28:42 2018 Train Epoch: 0 [1241600/1281167 (97%)]\tLoss: 6.900714\n",
"Thu Nov 1 04:29:05 2018 Train Epoch: 0 [1248000/1281167 (97%)]\tLoss: 6.916955\n",
"Thu Nov 1 04:29:28 2018 Train Epoch: 0 [1254400/1281167 (98%)]\tLoss: 6.914443\n",
"Thu Nov 1 04:29:50 2018 Train Epoch: 0 [1260800/1281167 (98%)]\tLoss: 6.917900\n",
"Thu Nov 1 04:30:13 2018 Train Epoch: 0 [1267200/1281167 (99%)]\tLoss: 6.898156\n",
"Thu Nov 1 04:30:35 2018 Train Epoch: 0 [1273600/1281167 (99%)]\tLoss: 6.915338\n",
"Thu Nov 1 04:30:57 2018 Train Epoch: 0 [1280000/1281167 (100%)]\tLoss: 6.908780\n",
"\n",
"Test set: Average loss: 6.9110, Accuracy: 50/50000 (0%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"Traceback (most recent call last):\n",
" File \"<ipython-input-29-cfc53eaa0ddf>\", line 40, in <module>\n",
" correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy =test(model, device, test_loader, False)\n",
"TypeError: 'NoneType' object is not iterable\n"
],
"name": "stderr"
}
]
},
{
"metadata": {
"id": "CHUI192YCNAN",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "5afbc4e5-d109-4fe0-c061-aed9c7f55aac"
},
"cell_type": "code",
"source": [
"#os.listdir('/gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/')\n",
"! curl --upload-file '/gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt' https://transfer.sh/000.pt\n",
"#os.remove('/gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt')"
],
"execution_count": 30,
"outputs": [
{
"output_type": "stream",
"text": [
"https://transfer.sh/mKZ4Y/000.pt"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "4-kI1JSzQbXB",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 7: Short answer questions\n",
"Please answer these questions, and put the answers in a file called homework1_python.pdf in your repository.\n",
"\n",
"# Tiny ImageNet\n",
"1. What design that you tried worked the best? This includes things like network design, learning rate, batch size, number of epochs, and other optimization parameters, data augmentation etc. What was the final train loss? Test loss? Test Accuracy? Provide the plots for train loss, test loss, and test accuracy.\n",
"2. What design worked the worst (but still performed better than random chance)? Provide all the same information as question 1.\n",
"3. Why do you think the best one worked well and the worst one worked poorly.\n",
"\n",
"\n",
"# Full ImageNet\n",
"1. What design that you tried worked the best? How many epochs were you able to run it for? Provide the same information from Tiny ImageNet question 1.\n",
"2. Were you able to use larger/deeper networks on Full ImageNet than you used on Tiny ImageNet and increase accuracy? If so, why? If not, why not?\n",
"3. The real ImageNet dataset has significantly larger images. How would you change your network design if the images were twice as large? How about smaller than Tiny ImageNet (32x32)? How do you think your accuracy would change? This is open-ended, but we want a more thought-out answer than \"I'd resize the images\" or \"I'd do a larger pooling stride.\" You don't have to write code to test your hypothesis.\n",
"\n",
"Additionally, download your .ipynb and put it in your repository and name it homework1_colab.ipynb\n",
"\n"
]
}
]
}stride=1)\n",
" self.conv10 = nn.Conv2d(128, 256, kernel_size=3, stride=1)\n",
" self.pool11 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv16 = nn.Conv2d(256, 512, kernel_size=3, stride=1)\n",
" self.pool17 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.avg4b4 = nn.AvgPool2d(kernel_size=5, stride=1, padding=0)\n",
" self.fc = nn.Linear(512, 256)\n",
" \n",
" \n",
" #use for save_best_model\n",
" self.best = 0.0\n",
" \n",
" def forward(self, x):# TODO define the forward pass\n",
" #raise NotImplementedError('Need to define the forward pass')\n",
" # dimms with batch size 10\n",
" x = x # [10, 3, 64, 64]\n",
" x = F.relu(self.conv4(x)) # [10, 128, 62, 62]\n",
" x = F.relu(self.conv5(x)) # [10, 64, 62, 62]\n",
" x = F.relu(self.conv6(x)) # [10, 128, 60, 60]\n",
" x = self.pool7(x) # [10, 128, 30, 30]\n",
" x = F.relu(self.conv8(x)) # [10, 256, 28, 28]\n",
" x = F.relu(self.conv9(x)) #[10, 128, 28, 28]\n",
" x = F.relu(self.conv10(x)) # [10, 256, 26, 26]\n",
" x = self.pool11(x) # [10, 256, 13, 13]\n",
" x = F.relu(self.conv16(x)) # [10, 512, 11, 11]\n",
" x = self.pool17(x) # [10, 512, 5, 5]\n",
" x = self.avg4b4(x) # [10, 512, 1, 1]\n",
" x = x.view(-1, 512)\n",
" x = self.fc(x)\n",
" \n",
"\n",
" return F.log_softmax(x, dim=1)\n",
"\n",
" def loss(self, prediction, label, reduction='elementwise_mean'):\n",
" loss_val = F.cross_entropy(prediction, label.squeeze(), reduction=reduction)\n",
" return loss_val\n",
"\n",
" def save_model(self, file_path, num_to_keep=1):\n",
" pt_util.save(self, file_path, num_to_keep)\n",
" \n",
" def save_best_model(self, accuracy, file_path, num_to_keep=1):\n",
" # TODO save the model if it is the best\n",
" if (accuracy > self.best):\n",
" self.best = accuracy;\n",
" pt_util.save(self, file_path, num_to_keep)\n",
" else:\n",
" print('Old model was better new: ' + str(accuracy) + 'old: ' + str(self.best))\n",
" #raise NotImplementedError('Need to implement save_best_model')\n",
"\n",
" def load_model(self, file_path):\n",
" pt_util.restore(self, file_path)\n",
"\n",
" def load_last_model(self, dir_path):\n",
" return pt_util.restore_latest(self, dir_path)\n",
" \n",
" \n"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "yiJSkXjiKpDL",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"This time we are giving you the train and test functions, but feel free to modify them if you want. \n",
"\n",
"You may need to return some additional information for the logging portion of this assignment.\n"
]
},
{
"metadata": {
"id": "pmuzixXrkuYs",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"import time\n",
"def train(model, device, train_loader, optimizer, epoch, log_interval):\n",
" model.train()\n",
" for batch_idx, (data, label) in enumerate(train_loader):\n",
" data, label = data.to(device), label.to(device)\n",
" optimizer.zero_grad()\n",
" output = model(data)\n",
" loss = model.loss(output, label)\n",
" loss.backward()\n",
" optimizer.step()\n",
" if batch_idx % log_interval == 0:\n",
" print('{} Train Epoch: {} [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
" time.ctime(time.time()),\n",
" epoch, batch_idx * len(data), len(train_loader.dataset),\n",
" 100. * batch_idx / len(train_loader), loss.item()))\n",
"\n",
"def test(model, device, test_loader, return_images=False, log_interval=None):\n",
" model.eval()\n",
" test_loss = 0\n",
" correct = 0\n",
"\n",
" correct_images = []\n",
" correct_values = []\n",
"\n",
" error_images = []\n",
" predicted_values = []\n",
" gt_values = []\n",
" with torch.no_grad():\n",
" for batch_idx, (data, label) in enumerate(test_loader):\n",
" data, label = data.to(device), label.to(device)\n",
" output = model(data)\n",
" test_loss_on = model.loss(output, label, reduction='sum').item()\n",
" test_loss += test_loss_on\n",
" pred = output.max(1)[1]\n",
" correct_mask = pred.eq(label.view_as(pred))\n",
" num_correct = correct_mask.sum().item()\n",
" correct += num_correct\n",
" if return_images:\n",
" if num_correct > 0:\n",
" correct_images.append(data[correct_mask, ...].data.cpu().numpy())\n",
" correct_value_data = label[correct_mask].data.cpu().numpy()[:, 0]\n",
" correct_values.append(correct_value_data)\n",
" if num_correct < len(label):\n",
" error_data = data[~correct_mask, ...].data.cpu().numpy()\n",
" error_images.append(error_data)\n",
" predicted_value_data = pred[~correct_mask].data.cpu().numpy()\n",
" predicted_values.append(predicted_value_data)\n",
" gt_value_data = label[~correct_mask].data.cpu().numpy()[:, 0]\n",
" gt_values.append(gt_value_data)\n",
" if log_interval is not None and batch_idx % log_interval == 0:\n",
" print('{} Test: [{}/{} ({:.0f}%)]\\tLoss: {:.6f}'.format(\n",
" time.ctime(time.time()),\n",
" batch_idx * len(data), len(test_loader.dataset),\n",
" 100. * batch_idx / len(test_loader), test_loss_on))\n",
" if return_images:\n",
" correct_images = np.concatenate(correct_images, axis=0)\n",
" error_images = np.concatenate(error_images, axis=0)\n",
" predicted_values = np.concatenate(predicted_values, axis=0)\n",
" correct_values = np.concatenate(correct_values, axis=0)\n",
" gt_values = np.concatenate(gt_values, axis=0)\n",
"\n",
" test_loss /= len(test_loader.dataset)\n",
" test_accuracy = 100. * correct / len(test_loader.dataset)\n",
"\n",
" print('\\nTest set: Average loss: {:.4f}, Accuracy: {}/{} ({:.0f}%)\\n'.format(\n",
" test_loss, correct, len(test_loader.dataset), test_accuracy))\n",
" if return_images:\n",
" return correct_images, correct_values, error_images, predicted_values, gt_values, test_accuracy"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "EwMDBwoCDRS_",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# PART 3: Loading Data\n",
"PyTorch has a nice interface for dealing with a variety of data. You can read a good tutorial here https://pytorch.org/tutorials/beginner/data_loading_tutorial.html\n",
"Your friendly neighborhood TAs have made it even easier by preprocessing the data into a nice format. The data you uploaded is stored using hdf5 files which can be acecces a lot like Numpy arrays using the h5py package. In each of the files, there is a \"dataset\" called 'images', and one called 'labels'. Read more about h5py here http://docs.h5py.org/en/latest/quick.html\n",
"\n",
"Speed hint: With small datasets, it is almost always a good idea to cache the data to disk rather than continually read from files.\n"
]
},
{
"metadata": {
"id": "igPTIlBtk2vu",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"\n",
"# Data loader\n",
"class H5Dataset(torch.utils.data.Dataset):\n",
" def __init__(self, h5_file, transform=None):\n",
" \n",
" # Implement data loading.\n",
" f = h5py.File(h5_file, 'r')\n",
" self.images = f['images'].value\n",
" self.labels = f['labels'].value\n",
" \n",
" #self.labels = np.array(self.hfile['labels'])\n",
" self.transform = transform\n",
" #raise NotImplementedError('Need to implement the data loading')\n",
" f.close()\n",
"\n",
" def __len__(self):\n",
" # Implement the length function\n",
" return self.images.shape[0]\n",
" \n",
"\n",
" def __getitem__(self, idx):\n",
" # implement the getitem function\n",
" # You should return a tuple of:\n",
" # a torch tensor containing single image in CxHxW format and\n",
" # the label as a single tensor scalar\n",
" #raise NotImplementedError('Need to implement the data loading')\n",
" \n",
" # best practice f = h5py.File() and then f[\"image\"][idx] in here\n",
" \n",
" # w and h may be swapped, but channel is first\n",
" data = torch.FloatTensor(self.images[idx]).permute(2, 0, 1)\n",
" label = torch.LongTensor(self.labels[idx])\n",
" if self.transform:\n",
" data = self.transform(data)\n",
" return (data, label)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "tfWvYzBaKcO7",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 4: Dataset Augmentation\n",
"In the MNIST assignment, we didn't do any data augmentation because MNIST is kind of easy.\n",
"\n",
"In this assignment, you may find that data augmentation helps you a lot (or possibly hurts your performance).\n",
"\n",
"You can find a bunch preimplemented here https://pytorch.org/docs/stable/torchvision/transforms.html and you can also do your own as seen in the tutorial from part 3.\n",
"\n",
"Play around with various data augmentations we will suggest some.\n",
"\n",
"- ToPILImage - This one is useful for a lot of the built in transforms which expect PIL images. \n",
"- RandomHorizontalFlip\n",
"- RandomResizedCrop\n",
"- ColorJitter\n",
"- RandomRotation\n",
"- Normalize\n",
"- Adding various types of noise\n",
"- ToTensor - PyTorch expects the output from the dataset to be a tensor in CxHxW format.\n",
"\n",
"\n",
"Note: You should be careful about which of these you apply to the test data. You usually don't want to apply noise to the test data, but you do want to normalize it in the same way for example.\n"
]
},
{
"metadata": {
"id": "-5JeXSx9LIx3",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"import torchvision\n",
"\n",
"train_transforms = None #transforms.Compose([transforms.ColorJitter(brightness=0.1, contrast=0.1, saturation=0.1, hue=0.1), transforms.ToTensor(),])\n",
"\n",
"test_transforms = None\n",
"\n",
"data_train = H5Dataset(DATA_PATH + 'train.h5', transform=train_transforms)\n",
"data_test = H5Dataset(DATA_PATH + 'val.h5', transform=test_transforms)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "piz_PoP-N5mK",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 5: Training the network\n",
"Generally, it is useful to see how your training is going. Often people print their loss to make sure it goes down and their accuracy to make sure it goes up. But pictures are better than words. So for this part, you should record and plot the training loss, test loss, and test accuracy (and whatever else you want). \n",
"\n",
"We have created a very simple logging interface which essentially just saves and restores files via pickle in pt_util. Saving and restoring log data is important if you end your run early and want to continue where you left off rather than starting over.\n",
"\n",
"We have also provided a plot function which can plot a single line graph. You can use it and plot each value independently, or change it to plot them all in one graph. \n",
"\n",
"\n",
"__Important note: Do not forget to title your graphs and label your axes. Plots are meaningless without a way to read them.__\n",
"\n",
"Second Note: It will be helpful for you when deciding what network structure, data augmentation, and such work to title the graphs accordingly so you remember.\n",
"Third Note: The default setup right now saves and restores the network weights from a single folder. When you modify network architectures, you may want to save the resulting files in different folders (with appropriate names).\n",
"\n",
"We also provided a function for showing some results, because it's not satisfying to train a neural net, you also want to see what it can do! This can also be useful for figuring out what your network is doing well, and what it is failing at. This type of error analysis is very common when training neural networks.\n"
]
},
{
"metadata": {
"id": "Hj-JBTfwk-4A",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Play around with these constants, you may find a better setting.\n",
"BATCH_SIZE = 256\n",
"TEST_BATCH_SIZE = 10\n",
"EPOCHS = 200\n",
"LEARNING_RATE = 0.01\n",
"MOMENTUM = 0.9\n",
"USE_CUDA = True\n",
"PRINT_INTERVAL = 100\n",
"WEIGHT_DECAY = 0.0005\n",
"LOG_PATH = DATA_PATH + 'log.pkl'\n",
"\n",
"# Now the actual training code\n",
"use_cuda = USE_CUDA and torch.cuda.is_available()\n",
"\n",
"device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
"print('Using device', device)\n",
"import multiprocessing\n",
"print('num cpus:', multiprocessing.cpu_count())\n",
"\n",
"kwargs = {'num_workers': multiprocessing.cpu_count(),\n",
" 'pin_memory': True} if use_cuda else {}\n",
"\n",
"class_names = [line.strip().split(', ') for line in open(DATA_PATH + 'class_names.txt')]\n",
"name_to_class = {line[1]: line[0] for line in class_names}\n",
"class_names = [line[1] for line in class_names]\n",
"\n",
"train_loader = torch.utils.data.DataLoader(data_train, batch_size=BATCH_SIZE,\n",
" shuffle=True, **kwargs)\n",
"test_loader = torch.utils.data.DataLoader(data_test, batch_size=TEST_BATCH_SIZE,\n",
" shuffle=False, **kwargs)\n",
"\n",
"model = TinyImagenetNet().to(device)\n",
"optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n",
"start_epoch = model.load_last_model(DATA_PATH + 'checkpoints')\n",
"\n",
"# You may want to define another default for your log data depending on how you save it.\n",
"log_data = pt_util.read_log(LOG_PATH, [])\n",
"\n",
"correct_images, correct_val, error_images, predicted_val, gt_val = test(model, device, test_loader, True)\n",
"correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
"error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
"pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
"pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])\n",
"\n",
"try:\n",
" for epoch in range(start_epoch, EPOCHS + 1):\n",
" train(model, device, train_loader, optimizer, epoch, PRINT_INTERVAL)\n",
" correct_images, correct_val, error_images, predicted_val, gt_val = test(model, device, test_loader, True)\n",
" # TODO define other things to do at the end of each loop like logging and saving the best model.\n",
"\n",
"\n",
"except KeyboardInterrupt as ke:\n",
" print('Interrupted')\n",
"except:\n",
" import traceback\n",
" traceback.print_exc()\n",
"finally:\n",
" # Always save the most recent model, but don't delete any existing ones.\n",
" model.save_model(DATA_PATH + 'checkpoints/%03d.pt' % epoch, 0)\n",
" \n",
" # Show some current correct/incorrect images.\n",
" correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
" error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
" pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
" pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])"
]
},
{
"metadata": {
"id": "Al5ltOeHvJdz",
"colab_type": "code",
"outputId": "b1a5c5ea-26fe-43e1-852e-480c39714ee1",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 3871
}
},
"cell_type": "code",
"source": [
"# Play around with these constants, you may find a better setting.\n",
"BATCH_SIZE = 256 # stock\n",
"#BATCH_SIZE = 1024 this caused an error\n",
"TEST_BATCH_SIZE = 10\n",
"EPOCHS = 10\n",
"#LEARNING_RATE = 0.01\n",
"LEARNING_RATE = 0.02\n",
"MOMENTUM = 0.9\n",
"USE_CUDA = True\n",
"PRINT_INTERVAL = 100\n",
"WEIGHT_DECAY = 0.0005\n",
"LOG_PATH = DATA_PATH + 'log.pkl'\n",
"\n",
"# Now the actual training code\n",
"use_cuda = USE_CUDA and torch.cuda.is_available()\n",
"\n",
"device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
"print('Using device', device)\n",
"import multiprocessing\n",
"print('num cpus:', multiprocessing.cpu_count())\n",
"\n",
"kwargs = {'num_workers': multiprocessing.cpu_count(),\n",
" 'pin_memory': True} if use_cuda else {}\n",
"\n",
"class_names = [line.strip().split(', ') for line in open(DATA_PATH + 'class_names.txt')]\n",
"name_to_class = {line[1]: line[0] for line in class_names}\n",
"class_names = [line[1] for line in class_names]\n",
"\n",
"train_loader = torch.utils.data.DataLoader(data_train, batch_size=BATCH_SIZE,\n",
" shuffle=True, **kwargs)\n",
"test_loader = torch.utils.data.DataLoader(data_test, batch_size=TEST_BATCH_SIZE,\n",
" shuffle=False, **kwargs)\n",
"\n",
"model = TinyImagenetNet().to(device)\n",
"optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n",
"start_epoch = model.load_last_model(DATA_PATH + 'checkpoints')\n",
"\n",
"# You may want to define another default for your log data depending on how you save it.\n",
"log_data = pt_util.read_log(LOG_PATH, [])\n",
"\n",
"correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy = test(model, device, test_loader, True)\n",
"correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
"error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
"pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
"print(min(predicted_val))\n",
"print(max(predicted_val))\n",
"print(min(gt_val))\n",
"print(max(gt_val))\n",
"print(len(class_names))\n",
"pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])\n",
"\n",
"try:\n",
" for epoch in range(start_epoch, EPOCHS + 1):\n",
" train(model, device, train_loader, optimizer, epoch, PRINT_INTERVAL)\n",
" correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy = test(model, device, test_loader, True)\n",
" # TODO define other things to do at the end of each loop like logging and saving the best model.\n",
" model.save_best_model(test_accuracy, DATA_PATH + 'checkpoints/best.pt')\n",
" \n",
"\n",
"except KeyboardInterrupt as ke:\n",
" print('Interrupted')\n",
"except:\n",
" import traceback\n",
" traceback.print_exc()\n",
"finally:\n",
" # Always save the most recent model, but don't delete any existing ones.\n",
" model.save_model(DATA_PATH + 'checkpoints/%03d.pt' % epoch, 0)\n",
" \n",
" # Show some current correct/incorrect images.\n",
" correct_images = pt_util.to_scaled_uint8(correct_images.transpose(0, 2, 3, 1))\n",
" error_images = pt_util.to_scaled_uint8(error_images.transpose(0, 2, 3, 1))\n",
" pt_util.show_images(correct_images, ['correct: %s' % class_names[aa] for aa in correct_val])\n",
" pt_util.show_images(error_images, ['pred: %s, actual: %s' % (class_names[aa], class_names[bb]) for aa, bb in zip(predicted_val, gt_val)])"
],
"execution_count": 55,
"outputs": [
{
"output_type": "stream",
"text": [
"Using device cuda\n",
"num cpus: 2\n",
"\n",
"Test set: Average loss: 5.5672, Accuracy: 48/10000 (0%)\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe11b830278>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "stream",
"text": [
"123\n",
"173\n",
"0\n",
"199\n",
"200\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe1192cbef0>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "stream",
"text": [
"Tue Oct 30 23:00:32 2018 Train Epoch: 0 [0/100000 (0%)]\tLoss: 5.549253\n",
"Tue Oct 30 23:01:56 2018 Train Epoch: 0 [25600/100000 (26%)]\tLoss: 5.179053\n",
"Tue Oct 30 23:03:20 2018 Train Epoch: 0 [51200/100000 (51%)]\tLoss: 5.062684\n",
"Tue Oct 30 23:04:43 2018 Train Epoch: 0 [76800/100000 (77%)]\tLoss: 4.937111\n",
"\n",
"Test set: Average loss: 4.8247, Accuracy: 340/10000 (3%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:06:21 2018 Train Epoch: 1 [0/100000 (0%)]\tLoss: 4.876338\n",
"Tue Oct 30 23:07:45 2018 Train Epoch: 1 [25600/100000 (26%)]\tLoss: 4.729456\n",
"Tue Oct 30 23:09:09 2018 Train Epoch: 1 [51200/100000 (51%)]\tLoss: 4.511452\n",
"Tue Oct 30 23:10:32 2018 Train Epoch: 1 [76800/100000 (77%)]\tLoss: 4.484566\n",
"\n",
"Test set: Average loss: 4.2662, Accuracy: 1017/10000 (10%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:12:10 2018 Train Epoch: 2 [0/100000 (0%)]\tLoss: 4.280207\n",
"Tue Oct 30 23:13:34 2018 Train Epoch: 2 [25600/100000 (26%)]\tLoss: 4.055193\n",
"Tue Oct 30 23:14:58 2018 Train Epoch: 2 [51200/100000 (51%)]\tLoss: 3.978329\n",
"Tue Oct 30 23:16:22 2018 Train Epoch: 2 [76800/100000 (77%)]\tLoss: 4.068361\n",
"\n",
"Test set: Average loss: 3.8912, Accuracy: 1558/10000 (16%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:18:01 2018 Train Epoch: 3 [0/100000 (0%)]\tLoss: 3.833221\n",
"Tue Oct 30 23:19:24 2018 Train Epoch: 3 [25600/100000 (26%)]\tLoss: 3.750987\n",
"Tue Oct 30 23:20:48 2018 Train Epoch: 3 [51200/100000 (51%)]\tLoss: 3.846452\n",
"Tue Oct 30 23:22:12 2018 Train Epoch: 3 [76800/100000 (77%)]\tLoss: 3.569184\n",
"\n",
"Test set: Average loss: 3.7381, Accuracy: 1882/10000 (19%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:23:50 2018 Train Epoch: 4 [0/100000 (0%)]\tLoss: 3.777778\n",
"Tue Oct 30 23:25:14 2018 Train Epoch: 4 [25600/100000 (26%)]\tLoss: 3.647930\n",
"Tue Oct 30 23:26:38 2018 Train Epoch: 4 [51200/100000 (51%)]\tLoss: 3.421569\n",
"Tue Oct 30 23:28:02 2018 Train Epoch: 4 [76800/100000 (77%)]\tLoss: 3.471924\n",
"\n",
"Test set: Average loss: 3.4881, Accuracy: 2269/10000 (23%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:29:40 2018 Train Epoch: 5 [0/100000 (0%)]\tLoss: 3.263522\n",
"Tue Oct 30 23:31:04 2018 Train Epoch: 5 [25600/100000 (26%)]\tLoss: 3.184380\n",
"Tue Oct 30 23:32:28 2018 Train Epoch: 5 [51200/100000 (51%)]\tLoss: 3.437448\n",
"Tue Oct 30 23:33:52 2018 Train Epoch: 5 [76800/100000 (77%)]\tLoss: 2.971600\n",
"\n",
"Test set: Average loss: 3.3469, Accuracy: 2580/10000 (26%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:35:31 2018 Train Epoch: 6 [0/100000 (0%)]\tLoss: 2.911456\n",
"Tue Oct 30 23:36:54 2018 Train Epoch: 6 [25600/100000 (26%)]\tLoss: 2.977590\n",
"Tue Oct 30 23:38:18 2018 Train Epoch: 6 [51200/100000 (51%)]\tLoss: 2.888722\n",
"Tue Oct 30 23:39:42 2018 Train Epoch: 6 [76800/100000 (77%)]\tLoss: 3.083384\n",
"\n",
"Test set: Average loss: 3.1720, Accuracy: 2861/10000 (29%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:41:21 2018 Train Epoch: 7 [0/100000 (0%)]\tLoss: 2.921553\n",
"Tue Oct 30 23:42:45 2018 Train Epoch: 7 [25600/100000 (26%)]\tLoss: 2.905390\n",
"Tue Oct 30 23:44:08 2018 Train Epoch: 7 [51200/100000 (51%)]\tLoss: 2.958408\n",
"Tue Oct 30 23:45:32 2018 Train Epoch: 7 [76800/100000 (77%)]\tLoss: 2.599775\n",
"\n",
"Test set: Average loss: 3.0303, Accuracy: 3185/10000 (32%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:47:11 2018 Train Epoch: 8 [0/100000 (0%)]\tLoss: 2.647141\n",
"Tue Oct 30 23:48:35 2018 Train Epoch: 8 [25600/100000 (26%)]\tLoss: 2.548127\n",
"Tue Oct 30 23:49:58 2018 Train Epoch: 8 [51200/100000 (51%)]\tLoss: 2.874490\n",
"Tue Oct 30 23:51:22 2018 Train Epoch: 8 [76800/100000 (77%)]\tLoss: 2.496460\n",
"\n",
"Test set: Average loss: 2.9749, Accuracy: 3340/10000 (33%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Tue Oct 30 23:53:01 2018 Train Epoch: 9 [0/100000 (0%)]\tLoss: 2.464377\n",
"Tue Oct 30 23:54:25 2018 Train Epoch: 9 [25600/100000 (26%)]\tLoss: 2.417069\n",
"Tue Oct 30 23:55:49 2018 Train Epoch: 9 [51200/100000 (51%)]\tLoss: 2.488547\n",
"Tue Oct 30 23:57:12 2018 Train Epoch: 9 [76800/100000 (77%)]\tLoss: 2.103152\n",
"\n",
"Test set: Average loss: 3.0223, Accuracy: 3240/10000 (32%)\n",
"\n",
"Old model was better new: 32.4old: 33.4\n",
"Tue Oct 30 23:58:51 2018 Train Epoch: 10 [0/100000 (0%)]\tLoss: 2.238087\n",
"Wed Oct 31 00:00:15 2018 Train Epoch: 10 [25600/100000 (26%)]\tLoss: 2.466787\n",
"Wed Oct 31 00:01:39 2018 Train Epoch: 10 [51200/100000 (51%)]\tLoss: 2.474845\n",
"Wed Oct 31 00:03:02 2018 Train Epoch: 10 [76800/100000 (77%)]\tLoss: 2.465969\n",
"\n",
"Test set: Average loss: 2.9739, Accuracy: 3461/10000 (35%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/010.pt\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe11bb69cc0>"
]
},
"metadata": {
"tags": []
}
},
{
"output_type": "display_data",
"data": {
"text/plain": [
"<matplotlib.figure.Figure at 0x7fe119456160>"
]
},
"metadata": {
"tags": []
}
}
]
},
{
"metadata": {
"id": "w9SP7tMAMalF",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "6369f07b-212b-46d8-bbbc-1c2c7af227c2"
},
"cell_type": "code",
"source": [
"os.listdir('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/000.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/005.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/007.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/004.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/010.pt')\n",
"# os.remove('/gdrive/My Drive/colab_files/homework1/tiny_imagenet/checkpoints/best.pt')"
],
"execution_count": 52,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"[]"
]
},
"metadata": {
"tags": []
},
"execution_count": 52
}
]
},
{
"metadata": {
"id": "4FGXjLjtFbHa",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 6: ImageNet!\n",
"ImageNet is the big granddaddy of supervised vision datasets. It's the dataset that got the whole deep learning train running. We thought it would be fun for you guys to try your hand at it.\n",
"\n",
"A few notes:\n",
"- Downloading ImageNet should take about 7 minutues. Extracting should take another 7 minutes. So run this code, then go have a coffee or whatever.\n",
"- The data will be removed after 24 hours or so. That's just how Colab works. Each time you reset the environment, you will also have to redownload ImageNet. \n",
" - I would recommend against storing it on your on Google Drive though as that is way slower to read during training.\n",
" - Through extensive testing, we have found that the fastest way to get the data is the code we provide (faster than saving to google drive, uploading by hand, etc.) \n",
"- Reading images directly with PIL seems to have issues on Colab. Instead try cv2.imread.\n",
" - cv2.imread returns images in bgr order. To go from bgr -> rgb, use `im = im[:, :, ::-1]`\n",
"- The images are provided such that the minimum side length is at least 128 unless the original image was too small for it to be shrunk.\n",
" - transforms.Resize will be your friend\n",
"- It seams you cannot use a batch size > 64 x 128 x 128 x 3 while still using multiprocessing due to Colab limitations. You can use larger batches if you set num_workers = 0, but it will be much slower.\n",
"- This dataset has 1000 classes, whereas Tiny ImageNet only had 200.\n",
"- You will probably want to save your results more than once every epoch since Colab may spurriously shut down your session.\n",
"- We know this takes a long time to run. Try leaving it overnight. We don't expect you to get state-of-the-art performance.\n",
"- `glob.glob` is a good Python function for reading lots of folder/file names, but it does not return a sorted list.\n",
"\n",
"\n"
]
},
{
"metadata": {
"id": "68XKUg9hF5Io",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 187
},
"outputId": "d70ccd0a-4d3f-4fa1-fff8-496a899669ff"
},
"cell_type": "code",
"source": [
"# Downloads and extracts the data\n",
"if not os.path.exists('imagenet128'):\n",
" !echo $(date +%x_%r)\n",
" !gsutil cp gs://imagenet-cropped/imagenet128.tar .\n",
" !echo $(date +%x_%r)\n",
" !tar -xf imagenet128.tar\n",
" !echo $(date +%x_%r)\n",
" !ls imagenet128/train | head -4"
],
"execution_count": 9,
"outputs": [
{
"output_type": "stream",
"text": [
"10/31/2018_08:39:48 PM\n",
"Copying gs://imagenet-cropped/imagenet128.tar...\n",
"\\ [1 files][ 23.7 GiB/ 23.7 GiB] 55.7 MiB/s \n",
"Operation completed over 1 objects/23.7 GiB. \n",
"10/31/2018_08:47:28 PM\n",
"10/31/2018_08:53:24 PM\n",
"n01440764\n",
"n01443537\n",
"n01484850\n",
"n01491361\n"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "-DqAvGsk89qt",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
" # Sets up synset data. Have a look at what is in these dictionaries as they may help you with debugging.\n",
"import json\n",
"imagenet_synset_data = json.load(open('imagenet128/imagenet_synsets.json'))\n",
"synset_id_to_cls = {val['id']: int(key) for key, val in imagenet_synset_data.items()}"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "4zD1uP7lW5ue",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"!export CUDA_LAUNCH_BLOCKING=0"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "0bUNNoLlGWgc",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"# Data loader\n",
"import cv2\n",
"class ImageFolderLoader(torch.utils.data.Dataset):\n",
" def __init__(self, folder, transform=None):\n",
" self.folder = folder\n",
" self.transform = transform\n",
" mydict = {}\n",
"\n",
" for directory in os.listdir(self.folder):\n",
" for file in os.listdir(self.folder + '/' + directory):\n",
" filename =self.folder + '/' + directory + '/' + file\n",
" mydict[filename] = synset_id_to_cls[directory]\n",
" \n",
" self.dictlist = []\n",
" for key, value in mydict.items():\n",
" temp = [key, value]\n",
" self.dictlist.append(temp)\n",
" del mydict\n",
" \n",
"\n",
" def __len__(self):\n",
" return len(self.dictlist)\n",
" def __getitem__(self, idx):\n",
" #dictlist is a list of tuples ('filename', classNumber)\n",
" filename = self.dictlist[idx][0]\n",
" label = self.dictlist[idx][1]\n",
" #cv2.imread returns images in bgr order. To go from bgr -> rgb, use im = im[:, :, ::-1]\n",
" im = cv2.imread(filename)\n",
" im = im[:, :, ::-1]\n",
" imtensor = torch.FloatTensor(im.copy()).permute((2, 0, 1))\n",
" #print('Pre transform' + str(imtensor.shape))\n",
" if self.transform:\n",
" imtensor = self.transform(imtensor)\n",
" #print('Post transform' + str(imtensor.shape))\n",
" \n",
" labeltensor = torch.LongTensor([label])\n",
" #print('Labeltensor' + str(labeltensor.shape))\n",
" return (imtensor, labeltensor)\n"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "IwBV9dcTGYqx",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"train_transforms = transforms.Compose([transforms.ToPILImage(), transforms.Resize((64,64)), transforms.ToTensor()])\n",
"test_transforms = transforms.Compose([transforms.ToPILImage(), transforms.Resize((64,64)), transforms.ToTensor()])\n",
"\n",
"data_train = ImageFolderLoader('imagenet128/train', transform=train_transforms)\n",
"assert(len(data_train) == 1281167)\n",
"data_test = ImageFolderLoader('imagenet128/val', transform=test_transforms)\n",
"assert(len(data_test) == 50000)"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "hp9pprc_-GEf",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"data_train[0]"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "ZAwdjaIuGj34",
"colab_type": "code",
"colab": {}
},
"cell_type": "code",
"source": [
"class FullImagenetNet(TinyImagenetNet):\n",
" def __init__(self):\n",
" super(FullImagenetNet, self).__init__()\n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # TODO define the layers\n",
" \n",
" # I think there are 3 input channels for 3 colors\n",
" # Lets try doing a combination of resnet nad darknet\n",
" \n",
" # now just using a sample of darknet\n",
" self.conv4 = nn.Conv2d(3, 128, kernel_size=3, stride=1)\n",
" self.conv5 = nn.Conv2d(128, 64, kernel_size=1, stride=1)\n",
" self.conv6 = nn.Conv2d(64, 128, kernel_size=3, stride=1)\n",
" self.pool7 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv8 = nn.Conv2d(128, 256, kernel_size=3, stride = 1)\n",
" self.conv9 = nn.Conv2d(256, 128, kernel_size=1, stride=1)\n",
" self.conv10 = nn.Conv2d(128, 256, kernel_size=3, stride=1)\n",
" self.pool11 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.conv16 = nn.Conv2d(256, 512, kernel_size=3, stride=1)\n",
" self.pool17 = nn.MaxPool2d(kernel_size=2, stride=2, padding=0)\n",
" self.avg4b4 = nn.AvgPool2d(kernel_size=5, stride=1, padding=0)\n",
" self.fc = nn.Linear(512, 1000)\n",
" \n",
" \n",
" #use for save_best_model\n",
" self.best = 0.0\n",
" \n",
" def forward(self, x):# TODO define the forward pass\n",
" #raise NotImplementedError('Need to define the forward pass')\n",
" # dimms with batch size 10\n",
" x = x # ([64, 3, 64, 64])\n",
" x = F.relu(self.conv4(x)) # [10, 128, 62, 62]\n",
" x = F.relu(self.conv5(x)) # [10, 64, 62, 62]\n",
" x = F.relu(self.conv6(x)) # [10, 128, 60, 60]\n",
" x = self.pool7(x) # [10, 128, 30, 30]\n",
" x = F.relu(self.conv8(x)) # [10, 256, 28, 28]\n",
" x = F.relu(self.conv9(x)) #[10, 128, 28, 28]\n",
" x = F.relu(self.conv10(x)) # [10, 256, 26, 26]\n",
" x = self.pool11(x) # [10, 256, 13, 13]\n",
" x = F.relu(self.conv16(x)) # [10, 512, 11, 11]\n",
" x = self.pool17(x) # [10, 512, 5, 5]\n",
" x = self.avg4b4(x) # [10, 512, 1, 1]\n",
" x = x.view(-1, 512)\n",
" x = self.fc(x)\n",
" \n",
" \n",
" return F.log_softmax(x, dim=1)\n"
],
"execution_count": 0,
"outputs": []
},
{
"metadata": {
"id": "tL0yjhP_Gnd0",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 3978
},
"outputId": "797280ec-2e96-4880-d310-b2ad9d790fce"
},
"cell_type": "code",
"source": [
"BATCH_SIZE = 64\n",
"TEST_BATCH_SIZE = 50\n",
"EPOCHS = 200\n",
"LEARNING_RATE = 0.1#0.01\n",
"MOMENTUM = 0.9\n",
"USE_CUDA = True\n",
"PRINT_INTERVAL = 100\n",
"WEIGHT_DECAY = 0.0005\n",
"CHECKPOINT_PATH = BASE_PATH + 'imagenet_full/checkpoints'\n",
"LOG_PATH = BASE_PATH + 'imagenet_full/' + 'log.pkl'\n",
"\n",
"use_cuda = USE_CUDA and torch.cuda.is_available()\n",
"\n",
"device = torch.device(\"cuda\" if use_cuda else \"cpu\")\n",
"print('Using device', device)\n",
"import multiprocessing\n",
"print('num cpus:', multiprocessing.cpu_count())\n",
"\n",
"kwargs = {'num_workers': multiprocessing.cpu_count(),\n",
" 'pin_memory': True} if use_cuda else {}\n",
"\n",
"\n",
"class_names = sorted([(int(key), val['label'].split(',')[0]) for key, val in imagenet_synset_data.items()])\n",
"name_to_class = {line[1]: line[0] for line in class_names}\n",
"class_names = [line[1] for line in class_names]\n",
"#print(data_train)\n",
"train_loader = torch.utils.data.DataLoader(data_train, batch_size=BATCH_SIZE,\n",
" shuffle=True, **kwargs)\n",
"test_loader = torch.utils.data.DataLoader(data_test, batch_size=TEST_BATCH_SIZE,\n",
" shuffle=False, **kwargs)\n",
"\n",
"model = FullImagenetNet().to(device)\n",
"optimizer = optim.SGD(model.parameters(), lr=LEARNING_RATE, momentum=MOMENTUM, weight_decay=WEIGHT_DECAY)\n",
"start_epoch = model.load_last_model(CHECKPOINT_PATH)\n",
"\n",
"\n",
"try:\n",
" for epoch in range(start_epoch, EPOCHS + 1):\n",
" train(model, device, train_loader, optimizer, epoch, PRINT_INTERVAL)\n",
" correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy =test(model, device, test_loader, False)\n",
" model.save_best_model(test_accuracy, CHECKPOINT_PATH + '/%03d.pt' % epoch)\n",
"\n",
"\n",
"except KeyboardInterrupt as ke:\n",
" print('Interrupted')\n",
"except:\n",
" import traceback\n",
" traceback.print_exc()\n",
"finally:\n",
" model.save_model(CHECKPOINT_PATH + '/%03d.pt' % epoch, 0)"
],
"execution_count": 29,
"outputs": [
{
"output_type": "stream",
"text": [
"Using device cuda\n",
"num cpus: 2\n",
"Restoring:\n",
"conv4.weight -> \ttorch.Size([128, 3, 3, 3]) = 0MB\n",
"conv4.bias -> \ttorch.Size([128]) = 0MB\n",
"conv5.weight -> \ttorch.Size([64, 128, 1, 1]) = 0MB\n",
"conv5.bias -> \ttorch.Size([64]) = 0MB\n",
"conv6.weight -> \ttorch.Size([128, 64, 3, 3]) = 0MB\n",
"conv6.bias -> \ttorch.Size([128]) = 0MB\n",
"conv8.weight -> \ttorch.Size([256, 128, 3, 3]) = 1MB\n",
"conv8.bias -> \ttorch.Size([256]) = 0MB\n",
"conv9.weight -> \ttorch.Size([128, 256, 1, 1]) = 0MB\n",
"conv9.bias -> \ttorch.Size([128]) = 0MB\n",
"conv10.weight -> \ttorch.Size([256, 128, 3, 3]) = 1MB\n",
"conv10.bias -> \ttorch.Size([256]) = 0MB\n",
"conv16.weight -> \ttorch.Size([512, 256, 3, 3]) = 4MB\n",
"conv16.bias -> \ttorch.Size([512]) = 0MB\n",
"fc.weight -> \ttorch.Size([1000, 512]) = 2MB\n",
"fc.bias -> \ttorch.Size([1000]) = 0MB\n",
"\n",
"Restored all variables\n",
"No new variables\n",
"Restored /gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt\n",
"Thu Nov 1 03:13:50 2018 Train Epoch: 0 [0/1281167 (0%)]\tLoss: 6.905530\n",
"Thu Nov 1 03:14:15 2018 Train Epoch: 0 [6400/1281167 (0%)]\tLoss: 6.915213\n",
"Thu Nov 1 03:14:39 2018 Train Epoch: 0 [12800/1281167 (1%)]\tLoss: 6.906976\n",
"Thu Nov 1 03:15:02 2018 Train Epoch: 0 [19200/1281167 (1%)]\tLoss: 6.903210\n",
"Thu Nov 1 03:15:26 2018 Train Epoch: 0 [25600/1281167 (2%)]\tLoss: 6.894549\n",
"Thu Nov 1 03:15:50 2018 Train Epoch: 0 [32000/1281167 (2%)]\tLoss: 6.913801\n",
"Thu Nov 1 03:16:13 2018 Train Epoch: 0 [38400/1281167 (3%)]\tLoss: 6.911386\n",
"Thu Nov 1 03:16:37 2018 Train Epoch: 0 [44800/1281167 (3%)]\tLoss: 6.903984\n",
"Thu Nov 1 03:17:00 2018 Train Epoch: 0 [51200/1281167 (4%)]\tLoss: 6.913480\n",
"Thu Nov 1 03:17:23 2018 Train Epoch: 0 [57600/1281167 (4%)]\tLoss: 6.921412\n",
"Thu Nov 1 03:17:47 2018 Train Epoch: 0 [64000/1281167 (5%)]\tLoss: 6.917922\n",
"Thu Nov 1 03:18:10 2018 Train Epoch: 0 [70400/1281167 (5%)]\tLoss: 6.917318\n",
"Thu Nov 1 03:18:34 2018 Train Epoch: 0 [76800/1281167 (6%)]\tLoss: 6.906056\n",
"Thu Nov 1 03:18:57 2018 Train Epoch: 0 [83200/1281167 (6%)]\tLoss: 6.903532\n",
"Thu Nov 1 03:19:21 2018 Train Epoch: 0 [89600/1281167 (7%)]\tLoss: 6.920099\n",
"Thu Nov 1 03:19:45 2018 Train Epoch: 0 [96000/1281167 (7%)]\tLoss: 6.896837\n",
"Thu Nov 1 03:20:08 2018 Train Epoch: 0 [102400/1281167 (8%)]\tLoss: 6.901331\n",
"Thu Nov 1 03:20:31 2018 Train Epoch: 0 [108800/1281167 (8%)]\tLoss: 6.912484\n",
"Thu Nov 1 03:20:54 2018 Train Epoch: 0 [115200/1281167 (9%)]\tLoss: 6.920592\n",
"Thu Nov 1 03:21:17 2018 Train Epoch: 0 [121600/1281167 (9%)]\tLoss: 6.903300\n",
"Thu Nov 1 03:21:41 2018 Train Epoch: 0 [128000/1281167 (10%)]\tLoss: 6.907892\n",
"Thu Nov 1 03:22:04 2018 Train Epoch: 0 [134400/1281167 (10%)]\tLoss: 6.911784\n",
"Thu Nov 1 03:22:27 2018 Train Epoch: 0 [140800/1281167 (11%)]\tLoss: 6.901532\n",
"Thu Nov 1 03:22:50 2018 Train Epoch: 0 [147200/1281167 (11%)]\tLoss: 6.914480\n",
"Thu Nov 1 03:23:14 2018 Train Epoch: 0 [153600/1281167 (12%)]\tLoss: 6.903480\n",
"Thu Nov 1 03:23:38 2018 Train Epoch: 0 [160000/1281167 (12%)]\tLoss: 6.903686\n",
"Thu Nov 1 03:24:01 2018 Train Epoch: 0 [166400/1281167 (13%)]\tLoss: 6.924271\n",
"Thu Nov 1 03:24:25 2018 Train Epoch: 0 [172800/1281167 (13%)]\tLoss: 6.912843\n",
"Thu Nov 1 03:24:50 2018 Train Epoch: 0 [179200/1281167 (14%)]\tLoss: 6.898242\n",
"Thu Nov 1 03:25:13 2018 Train Epoch: 0 [185600/1281167 (14%)]\tLoss: 6.926912\n",
"Thu Nov 1 03:25:36 2018 Train Epoch: 0 [192000/1281167 (15%)]\tLoss: 6.918694\n",
"Thu Nov 1 03:25:59 2018 Train Epoch: 0 [198400/1281167 (15%)]\tLoss: 6.920316\n",
"Thu Nov 1 03:26:22 2018 Train Epoch: 0 [204800/1281167 (16%)]\tLoss: 6.907867\n",
"Thu Nov 1 03:26:45 2018 Train Epoch: 0 [211200/1281167 (16%)]\tLoss: 6.905494\n",
"Thu Nov 1 03:27:08 2018 Train Epoch: 0 [217600/1281167 (17%)]\tLoss: 6.919506\n",
"Thu Nov 1 03:27:31 2018 Train Epoch: 0 [224000/1281167 (17%)]\tLoss: 6.919896\n",
"Thu Nov 1 03:27:55 2018 Train Epoch: 0 [230400/1281167 (18%)]\tLoss: 6.913472\n",
"Thu Nov 1 03:28:17 2018 Train Epoch: 0 [236800/1281167 (18%)]\tLoss: 6.911576\n",
"Thu Nov 1 03:28:40 2018 Train Epoch: 0 [243200/1281167 (19%)]\tLoss: 6.910405\n",
"Thu Nov 1 03:29:04 2018 Train Epoch: 0 [249600/1281167 (19%)]\tLoss: 6.914724\n",
"Thu Nov 1 03:29:27 2018 Train Epoch: 0 [256000/1281167 (20%)]\tLoss: 6.914860\n",
"Thu Nov 1 03:29:50 2018 Train Epoch: 0 [262400/1281167 (20%)]\tLoss: 6.917733\n",
"Thu Nov 1 03:30:13 2018 Train Epoch: 0 [268800/1281167 (21%)]\tLoss: 6.916535\n",
"Thu Nov 1 03:30:37 2018 Train Epoch: 0 [275200/1281167 (21%)]\tLoss: 6.895416\n",
"Thu Nov 1 03:31:00 2018 Train Epoch: 0 [281600/1281167 (22%)]\tLoss: 6.915490\n",
"Thu Nov 1 03:31:24 2018 Train Epoch: 0 [288000/1281167 (22%)]\tLoss: 6.927084\n",
"Thu Nov 1 03:31:48 2018 Train Epoch: 0 [294400/1281167 (23%)]\tLoss: 6.900946\n",
"Thu Nov 1 03:32:12 2018 Train Epoch: 0 [300800/1281167 (23%)]\tLoss: 6.930200\n",
"Thu Nov 1 03:32:35 2018 Train Epoch: 0 [307200/1281167 (24%)]\tLoss: 6.912046\n",
"Thu Nov 1 03:32:58 2018 Train Epoch: 0 [313600/1281167 (24%)]\tLoss: 6.917764\n",
"Thu Nov 1 03:33:22 2018 Train Epoch: 0 [320000/1281167 (25%)]\tLoss: 6.892279\n",
"Thu Nov 1 03:33:47 2018 Train Epoch: 0 [326400/1281167 (25%)]\tLoss: 6.924290\n",
"Thu Nov 1 03:34:11 2018 Train Epoch: 0 [332800/1281167 (26%)]\tLoss: 6.902865\n",
"Thu Nov 1 03:34:34 2018 Train Epoch: 0 [339200/1281167 (26%)]\tLoss: 6.913850\n",
"Thu Nov 1 03:34:57 2018 Train Epoch: 0 [345600/1281167 (27%)]\tLoss: 6.907212\n",
"Thu Nov 1 03:35:21 2018 Train Epoch: 0 [352000/1281167 (27%)]\tLoss: 6.916387\n",
"Thu Nov 1 03:35:44 2018 Train Epoch: 0 [358400/1281167 (28%)]\tLoss: 6.928115\n",
"Thu Nov 1 03:36:06 2018 Train Epoch: 0 [364800/1281167 (28%)]\tLoss: 6.894051\n",
"Thu Nov 1 03:36:29 2018 Train Epoch: 0 [371200/1281167 (29%)]\tLoss: 6.889081\n",
"Thu Nov 1 03:36:52 2018 Train Epoch: 0 [377600/1281167 (29%)]\tLoss: 6.902927\n",
"Thu Nov 1 03:37:17 2018 Train Epoch: 0 [384000/1281167 (30%)]\tLoss: 6.913784\n",
"Thu Nov 1 03:37:40 2018 Train Epoch: 0 [390400/1281167 (30%)]\tLoss: 6.917893\n",
"Thu Nov 1 03:38:03 2018 Train Epoch: 0 [396800/1281167 (31%)]\tLoss: 6.917501\n",
"Thu Nov 1 03:38:26 2018 Train Epoch: 0 [403200/1281167 (31%)]\tLoss: 6.906955\n",
"Thu Nov 1 03:38:49 2018 Train Epoch: 0 [409600/1281167 (32%)]\tLoss: 6.915683\n",
"Thu Nov 1 03:39:12 2018 Train Epoch: 0 [416000/1281167 (32%)]\tLoss: 6.906111\n",
"Thu Nov 1 03:39:35 2018 Train Epoch: 0 [422400/1281167 (33%)]\tLoss: 6.911583\n",
"Thu Nov 1 03:39:59 2018 Train Epoch: 0 [428800/1281167 (33%)]\tLoss: 6.915631\n",
"Thu Nov 1 03:40:22 2018 Train Epoch: 0 [435200/1281167 (34%)]\tLoss: 6.881934\n",
"Thu Nov 1 03:40:46 2018 Train Epoch: 0 [441600/1281167 (34%)]\tLoss: 6.905287\n",
"Thu Nov 1 03:41:09 2018 Train Epoch: 0 [448000/1281167 (35%)]\tLoss: 6.906995\n",
"Thu Nov 1 03:41:33 2018 Train Epoch: 0 [454400/1281167 (35%)]\tLoss: 6.905661\n",
"Thu Nov 1 03:41:56 2018 Train Epoch: 0 [460800/1281167 (36%)]\tLoss: 6.927936\n",
"Thu Nov 1 03:42:22 2018 Train Epoch: 0 [467200/1281167 (36%)]\tLoss: 6.908838\n",
"Thu Nov 1 03:42:48 2018 Train Epoch: 0 [473600/1281167 (37%)]\tLoss: 6.901138\n",
"Thu Nov 1 03:43:12 2018 Train Epoch: 0 [480000/1281167 (37%)]\tLoss: 6.915953\n",
"Thu Nov 1 03:43:35 2018 Train Epoch: 0 [486400/1281167 (38%)]\tLoss: 6.907689\n",
"Thu Nov 1 03:43:59 2018 Train Epoch: 0 [492800/1281167 (38%)]\tLoss: 6.914100\n",
"Thu Nov 1 03:44:22 2018 Train Epoch: 0 [499200/1281167 (39%)]\tLoss: 6.915280\n",
"Thu Nov 1 03:44:45 2018 Train Epoch: 0 [505600/1281167 (39%)]\tLoss: 6.920105\n",
"Thu Nov 1 03:45:09 2018 Train Epoch: 0 [512000/1281167 (40%)]\tLoss: 6.903097\n",
"Thu Nov 1 03:45:32 2018 Train Epoch: 0 [518400/1281167 (40%)]\tLoss: 6.907869\n",
"Thu Nov 1 03:45:55 2018 Train Epoch: 0 [524800/1281167 (41%)]\tLoss: 6.905593\n",
"Thu Nov 1 03:46:18 2018 Train Epoch: 0 [531200/1281167 (41%)]\tLoss: 6.909816\n",
"Thu Nov 1 03:46:42 2018 Train Epoch: 0 [537600/1281167 (42%)]\tLoss: 6.896438\n",
"Thu Nov 1 03:47:05 2018 Train Epoch: 0 [544000/1281167 (42%)]\tLoss: 6.907209\n",
"Thu Nov 1 03:47:28 2018 Train Epoch: 0 [550400/1281167 (43%)]\tLoss: 6.908332\n",
"Thu Nov 1 03:47:52 2018 Train Epoch: 0 [556800/1281167 (43%)]\tLoss: 6.906932\n",
"Thu Nov 1 03:48:14 2018 Train Epoch: 0 [563200/1281167 (44%)]\tLoss: 6.910619\n",
"Thu Nov 1 03:48:38 2018 Train Epoch: 0 [569600/1281167 (44%)]\tLoss: 6.903950\n",
"Thu Nov 1 03:49:01 2018 Train Epoch: 0 [576000/1281167 (45%)]\tLoss: 6.917201\n",
"Thu Nov 1 03:49:24 2018 Train Epoch: 0 [582400/1281167 (45%)]\tLoss: 6.899111\n",
"Thu Nov 1 03:49:46 2018 Train Epoch: 0 [588800/1281167 (46%)]\tLoss: 6.891315\n",
"Thu Nov 1 03:50:10 2018 Train Epoch: 0 [595200/1281167 (46%)]\tLoss: 6.913540\n",
"Thu Nov 1 03:50:33 2018 Train Epoch: 0 [601600/1281167 (47%)]\tLoss: 6.918234\n",
"Thu Nov 1 03:50:56 2018 Train Epoch: 0 [608000/1281167 (47%)]\tLoss: 6.923501\n",
"Thu Nov 1 03:51:18 2018 Train Epoch: 0 [614400/1281167 (48%)]\tLoss: 6.924489\n",
"Thu Nov 1 03:51:41 2018 Train Epoch: 0 [620800/1281167 (48%)]\tLoss: 6.910414\n",
"Thu Nov 1 03:52:04 2018 Train Epoch: 0 [627200/1281167 (49%)]\tLoss: 6.900335\n",
"Thu Nov 1 03:52:27 2018 Train Epoch: 0 [633600/1281167 (49%)]\tLoss: 6.915105\n",
"Thu Nov 1 03:52:51 2018 Train Epoch: 0 [640000/1281167 (50%)]\tLoss: 6.904331\n",
"Thu Nov 1 03:53:14 2018 Train Epoch: 0 [646400/1281167 (50%)]\tLoss: 6.920506\n",
"Thu Nov 1 03:53:37 2018 Train Epoch: 0 [652800/1281167 (51%)]\tLoss: 6.912586\n",
"Thu Nov 1 03:54:00 2018 Train Epoch: 0 [659200/1281167 (51%)]\tLoss: 6.905747\n",
"Thu Nov 1 03:54:23 2018 Train Epoch: 0 [665600/1281167 (52%)]\tLoss: 6.919553\n",
"Thu Nov 1 03:54:46 2018 Train Epoch: 0 [672000/1281167 (52%)]\tLoss: 6.901967\n",
"Thu Nov 1 03:55:09 2018 Train Epoch: 0 [678400/1281167 (53%)]\tLoss: 6.899582\n",
"Thu Nov 1 03:55:32 2018 Train Epoch: 0 [684800/1281167 (53%)]\tLoss: 6.907303\n",
"Thu Nov 1 03:55:55 2018 Train Epoch: 0 [691200/1281167 (54%)]\tLoss: 6.910735\n",
"Thu Nov 1 03:56:18 2018 Train Epoch: 0 [697600/1281167 (54%)]\tLoss: 6.902495\n",
"Thu Nov 1 03:56:40 2018 Train Epoch: 0 [704000/1281167 (55%)]\tLoss: 6.907861\n",
"Thu Nov 1 03:57:03 2018 Train Epoch: 0 [710400/1281167 (55%)]\tLoss: 6.905015\n",
"Thu Nov 1 03:57:26 2018 Train Epoch: 0 [716800/1281167 (56%)]\tLoss: 6.908472\n",
"Thu Nov 1 03:57:50 2018 Train Epoch: 0 [723200/1281167 (56%)]\tLoss: 6.898434\n",
"Thu Nov 1 03:58:14 2018 Train Epoch: 0 [729600/1281167 (57%)]\tLoss: 6.901069\n",
"Thu Nov 1 03:58:38 2018 Train Epoch: 0 [736000/1281167 (57%)]\tLoss: 6.892691\n",
"Thu Nov 1 03:59:03 2018 Train Epoch: 0 [742400/1281167 (58%)]\tLoss: 6.921463\n",
"Thu Nov 1 03:59:26 2018 Train Epoch: 0 [748800/1281167 (58%)]\tLoss: 6.902192\n",
"Thu Nov 1 03:59:50 2018 Train Epoch: 0 [755200/1281167 (59%)]\tLoss: 6.899998\n",
"Thu Nov 1 04:00:14 2018 Train Epoch: 0 [761600/1281167 (59%)]\tLoss: 6.895144\n",
"Thu Nov 1 04:00:38 2018 Train Epoch: 0 [768000/1281167 (60%)]\tLoss: 6.902062\n",
"Thu Nov 1 04:01:04 2018 Train Epoch: 0 [774400/1281167 (60%)]\tLoss: 6.901936\n",
"Thu Nov 1 04:01:27 2018 Train Epoch: 0 [780800/1281167 (61%)]\tLoss: 6.903090\n",
"Thu Nov 1 04:01:51 2018 Train Epoch: 0 [787200/1281167 (61%)]\tLoss: 6.904884\n",
"Thu Nov 1 04:02:15 2018 Train Epoch: 0 [793600/1281167 (62%)]\tLoss: 6.923146\n",
"Thu Nov 1 04:02:38 2018 Train Epoch: 0 [800000/1281167 (62%)]\tLoss: 6.913280\n",
"Thu Nov 1 04:03:02 2018 Train Epoch: 0 [806400/1281167 (63%)]\tLoss: 6.906809\n",
"Thu Nov 1 04:03:25 2018 Train Epoch: 0 [812800/1281167 (63%)]\tLoss: 6.913114\n",
"Thu Nov 1 04:03:48 2018 Train Epoch: 0 [819200/1281167 (64%)]\tLoss: 6.915143\n",
"Thu Nov 1 04:04:10 2018 Train Epoch: 0 [825600/1281167 (64%)]\tLoss: 6.921178\n",
"Thu Nov 1 04:04:33 2018 Train Epoch: 0 [832000/1281167 (65%)]\tLoss: 6.901847\n",
"Thu Nov 1 04:04:56 2018 Train Epoch: 0 [838400/1281167 (65%)]\tLoss: 6.938429\n",
"Thu Nov 1 04:05:19 2018 Train Epoch: 0 [844800/1281167 (66%)]\tLoss: 6.895684\n",
"Thu Nov 1 04:05:41 2018 Train Epoch: 0 [851200/1281167 (66%)]\tLoss: 6.901691\n",
"Thu Nov 1 04:06:04 2018 Train Epoch: 0 [857600/1281167 (67%)]\tLoss: 6.911649\n",
"Thu Nov 1 04:06:27 2018 Train Epoch: 0 [864000/1281167 (67%)]\tLoss: 6.923692\n",
"Thu Nov 1 04:06:50 2018 Train Epoch: 0 [870400/1281167 (68%)]\tLoss: 6.900036\n",
"Thu Nov 1 04:07:13 2018 Train Epoch: 0 [876800/1281167 (68%)]\tLoss: 6.927470\n",
"Thu Nov 1 04:07:35 2018 Train Epoch: 0 [883200/1281167 (69%)]\tLoss: 6.910205\n",
"Thu Nov 1 04:07:58 2018 Train Epoch: 0 [889600/1281167 (69%)]\tLoss: 6.874023\n",
"Thu Nov 1 04:08:21 2018 Train Epoch: 0 [896000/1281167 (70%)]\tLoss: 6.903159\n",
"Thu Nov 1 04:08:44 2018 Train Epoch: 0 [902400/1281167 (70%)]\tLoss: 6.908606\n",
"Thu Nov 1 04:09:07 2018 Train Epoch: 0 [908800/1281167 (71%)]\tLoss: 6.887091\n",
"Thu Nov 1 04:09:30 2018 Train Epoch: 0 [915200/1281167 (71%)]\tLoss: 6.902290\n",
"Thu Nov 1 04:09:53 2018 Train Epoch: 0 [921600/1281167 (72%)]\tLoss: 6.919454\n",
"Thu Nov 1 04:10:15 2018 Train Epoch: 0 [928000/1281167 (72%)]\tLoss: 6.910271\n",
"Thu Nov 1 04:10:38 2018 Train Epoch: 0 [934400/1281167 (73%)]\tLoss: 6.900114\n",
"Thu Nov 1 04:11:01 2018 Train Epoch: 0 [940800/1281167 (73%)]\tLoss: 6.925539\n",
"Thu Nov 1 04:11:24 2018 Train Epoch: 0 [947200/1281167 (74%)]\tLoss: 6.923165\n",
"Thu Nov 1 04:11:47 2018 Train Epoch: 0 [953600/1281167 (74%)]\tLoss: 6.901720\n",
"Thu Nov 1 04:12:10 2018 Train Epoch: 0 [960000/1281167 (75%)]\tLoss: 6.907963\n",
"Thu Nov 1 04:12:32 2018 Train Epoch: 0 [966400/1281167 (75%)]\tLoss: 6.921975\n",
"Thu Nov 1 04:12:54 2018 Train Epoch: 0 [972800/1281167 (76%)]\tLoss: 6.915666\n",
"Thu Nov 1 04:13:17 2018 Train Epoch: 0 [979200/1281167 (76%)]\tLoss: 6.914638\n",
"Thu Nov 1 04:13:40 2018 Train Epoch: 0 [985600/1281167 (77%)]\tLoss: 6.900302\n",
"Thu Nov 1 04:14:02 2018 Train Epoch: 0 [992000/1281167 (77%)]\tLoss: 6.907050\n",
"Thu Nov 1 04:14:25 2018 Train Epoch: 0 [998400/1281167 (78%)]\tLoss: 6.919683\n",
"Thu Nov 1 04:14:48 2018 Train Epoch: 0 [1004800/1281167 (78%)]\tLoss: 6.910315\n",
"Thu Nov 1 04:15:10 2018 Train Epoch: 0 [1011200/1281167 (79%)]\tLoss: 6.920113\n",
"Thu Nov 1 04:15:33 2018 Train Epoch: 0 [1017600/1281167 (79%)]\tLoss: 6.906448\n",
"Thu Nov 1 04:15:56 2018 Train Epoch: 0 [1024000/1281167 (80%)]\tLoss: 6.919313\n",
"Thu Nov 1 04:16:18 2018 Train Epoch: 0 [1030400/1281167 (80%)]\tLoss: 6.911633\n",
"Thu Nov 1 04:16:40 2018 Train Epoch: 0 [1036800/1281167 (81%)]\tLoss: 6.914162\n",
"Thu Nov 1 04:17:03 2018 Train Epoch: 0 [1043200/1281167 (81%)]\tLoss: 6.921151\n",
"Thu Nov 1 04:17:25 2018 Train Epoch: 0 [1049600/1281167 (82%)]\tLoss: 6.910539\n",
"Thu Nov 1 04:17:48 2018 Train Epoch: 0 [1056000/1281167 (82%)]\tLoss: 6.919050\n",
"Thu Nov 1 04:18:10 2018 Train Epoch: 0 [1062400/1281167 (83%)]\tLoss: 6.908661\n",
"Thu Nov 1 04:18:33 2018 Train Epoch: 0 [1068800/1281167 (83%)]\tLoss: 6.904510\n",
"Thu Nov 1 04:18:55 2018 Train Epoch: 0 [1075200/1281167 (84%)]\tLoss: 6.905267\n",
"Thu Nov 1 04:19:18 2018 Train Epoch: 0 [1081600/1281167 (84%)]\tLoss: 6.894763\n",
"Thu Nov 1 04:19:41 2018 Train Epoch: 0 [1088000/1281167 (85%)]\tLoss: 6.916194\n",
"Thu Nov 1 04:20:03 2018 Train Epoch: 0 [1094400/1281167 (85%)]\tLoss: 6.909140\n",
"Thu Nov 1 04:20:25 2018 Train Epoch: 0 [1100800/1281167 (86%)]\tLoss: 6.924272\n",
"Thu Nov 1 04:20:48 2018 Train Epoch: 0 [1107200/1281167 (86%)]\tLoss: 6.893477\n",
"Thu Nov 1 04:21:11 2018 Train Epoch: 0 [1113600/1281167 (87%)]\tLoss: 6.909297\n",
"Thu Nov 1 04:21:34 2018 Train Epoch: 0 [1120000/1281167 (87%)]\tLoss: 6.903625\n",
"Thu Nov 1 04:21:56 2018 Train Epoch: 0 [1126400/1281167 (88%)]\tLoss: 6.898579\n",
"Thu Nov 1 04:22:19 2018 Train Epoch: 0 [1132800/1281167 (88%)]\tLoss: 6.906099\n",
"Thu Nov 1 04:22:41 2018 Train Epoch: 0 [1139200/1281167 (89%)]\tLoss: 6.911266\n",
"Thu Nov 1 04:23:04 2018 Train Epoch: 0 [1145600/1281167 (89%)]\tLoss: 6.913440\n",
"Thu Nov 1 04:23:27 2018 Train Epoch: 0 [1152000/1281167 (90%)]\tLoss: 6.904614\n",
"Thu Nov 1 04:23:50 2018 Train Epoch: 0 [1158400/1281167 (90%)]\tLoss: 6.912544\n",
"Thu Nov 1 04:24:12 2018 Train Epoch: 0 [1164800/1281167 (91%)]\tLoss: 6.887463\n",
"Thu Nov 1 04:24:35 2018 Train Epoch: 0 [1171200/1281167 (91%)]\tLoss: 6.907408\n",
"Thu Nov 1 04:24:57 2018 Train Epoch: 0 [1177600/1281167 (92%)]\tLoss: 6.901124\n",
"Thu Nov 1 04:25:20 2018 Train Epoch: 0 [1184000/1281167 (92%)]\tLoss: 6.918556\n",
"Thu Nov 1 04:25:43 2018 Train Epoch: 0 [1190400/1281167 (93%)]\tLoss: 6.919122\n",
"Thu Nov 1 04:26:05 2018 Train Epoch: 0 [1196800/1281167 (93%)]\tLoss: 6.904559\n",
"Thu Nov 1 04:26:28 2018 Train Epoch: 0 [1203200/1281167 (94%)]\tLoss: 6.905938\n",
"Thu Nov 1 04:26:50 2018 Train Epoch: 0 [1209600/1281167 (94%)]\tLoss: 6.915270\n",
"Thu Nov 1 04:27:12 2018 Train Epoch: 0 [1216000/1281167 (95%)]\tLoss: 6.896585\n",
"Thu Nov 1 04:27:35 2018 Train Epoch: 0 [1222400/1281167 (95%)]\tLoss: 6.910329\n",
"Thu Nov 1 04:27:57 2018 Train Epoch: 0 [1228800/1281167 (96%)]\tLoss: 6.909390\n",
"Thu Nov 1 04:28:19 2018 Train Epoch: 0 [1235200/1281167 (96%)]\tLoss: 6.909443\n",
"Thu Nov 1 04:28:42 2018 Train Epoch: 0 [1241600/1281167 (97%)]\tLoss: 6.900714\n",
"Thu Nov 1 04:29:05 2018 Train Epoch: 0 [1248000/1281167 (97%)]\tLoss: 6.916955\n",
"Thu Nov 1 04:29:28 2018 Train Epoch: 0 [1254400/1281167 (98%)]\tLoss: 6.914443\n",
"Thu Nov 1 04:29:50 2018 Train Epoch: 0 [1260800/1281167 (98%)]\tLoss: 6.917900\n",
"Thu Nov 1 04:30:13 2018 Train Epoch: 0 [1267200/1281167 (99%)]\tLoss: 6.898156\n",
"Thu Nov 1 04:30:35 2018 Train Epoch: 0 [1273600/1281167 (99%)]\tLoss: 6.915338\n",
"Thu Nov 1 04:30:57 2018 Train Epoch: 0 [1280000/1281167 (100%)]\tLoss: 6.908780\n",
"\n",
"Test set: Average loss: 6.9110, Accuracy: 50/50000 (0%)\n",
"\n",
"Saved /gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt\n",
"\n"
],
"name": "stdout"
},
{
"output_type": "stream",
"text": [
"Traceback (most recent call last):\n",
" File \"<ipython-input-29-cfc53eaa0ddf>\", line 40, in <module>\n",
" correct_images, correct_val, error_images, predicted_val, gt_val, test_accuracy =test(model, device, test_loader, False)\n",
"TypeError: 'NoneType' object is not iterable\n"
],
"name": "stderr"
}
]
},
{
"metadata": {
"id": "CHUI192YCNAN",
"colab_type": "code",
"colab": {
"base_uri": "https://localhost:8080/",
"height": 34
},
"outputId": "5afbc4e5-d109-4fe0-c061-aed9c7f55aac"
},
"cell_type": "code",
"source": [
"#os.listdir('/gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/')\n",
"! curl --upload-file '/gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt' https://transfer.sh/000.pt\n",
"#os.remove('/gdrive/My Drive/colab_files/homework1/imagenet_full/checkpoints/000.pt')"
],
"execution_count": 30,
"outputs": [
{
"output_type": "stream",
"text": [
"https://transfer.sh/mKZ4Y/000.pt"
],
"name": "stdout"
}
]
},
{
"metadata": {
"id": "4-kI1JSzQbXB",
"colab_type": "text"
},
"cell_type": "markdown",
"source": [
"# Part 7: Short answer questions\n",
"Please answer these questions, and put the answers in a file called homework1_python.pdf in your repository.\n",
"\n",
"# Tiny ImageNet\n",
"1. What design that you tried worked the best? This includes things like network design, learning rate, batch size, number of epochs, and other optimization parameters, data augmentation etc. What was the final train loss? Test loss? Test Accuracy? Provide the plots for train loss, test loss, and test accuracy.\n",
"2. What design worked the worst (but still performed better than random chance)? Provide all the same information as question 1.\n",
"3. Why do you think the best one worked well and the worst one worked poorly.\n",
"\n",
"\n",
"# Full ImageNet\n",
"1. What design that you tried worked the best? How many epochs were you able to run it for? Provide the same information from Tiny ImageNet question 1.\n",
"2. Were you able to use larger/deeper networks on Full ImageNet than you used on Tiny ImageNet and increase accuracy? If so, why? If not, why not?\n",
"3. The real ImageNet dataset has significantly larger images. How would you change your network design if the images were twice as large? How about smaller than Tiny ImageNet (32x32)? How do you think your accuracy would change? This is open-ended, but we want a more thought-out answer than \"I'd resize the images\" or \"I'd do a larger pooling stride.\" You don't have to write code to test your hypothesis.\n",
"\n",
"Additionally, download your .ipynb and put it in your repository and name it homework1_colab.ipynb\n",
"\n"
]
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment