Skip to content

Instantly share code, notes, and snippets.

@adpoe
Last active December 24, 2016 01:47
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save adpoe/12ccf0f4498aa0377a16327c5dea4175 to your computer and use it in GitHub Desktop.
Save adpoe/12ccf0f4498aa0377a16327c5dea4175 to your computer and use it in GitHub Desktop.
A Quick Start Guide for Using Keras in the Nature Conservatory Kaggle
Display the source blob
Display the rendered blob
Raw
{
"metadata": {
"kernelspec": {
"name": "python"
},
"language_info": {
"name": "python",
"version": "3.5.1"
}
},
"nbformat": 4,
"nbformat_minor": 0,
"cells": [
{
"cell_type": "markdown",
"source": "# First Steps with Keras Convolutional Neural Networks - Nature Conservancy Image Recognition Kaggle",
"metadata": {}
},
{
"cell_type": "markdown",
"source": "**Keras CNN that will yield 95% accuracy on its training data, if you use the full data set, 25+ epochs.** (Here we use a subset, and only a few epochs, for sake of speed.) More training epochs and more + better data --> more accuracy.\n\nThis CNN was created using public tutorials, but updated to work on the data set for the current project.",
"metadata": {}
},
{
"cell_type": "markdown",
"source": " **For sources, see:**\n\n - [http://machinelearningmastery.com/object-recognition-convolutional-neural-networks-keras-deep-learning-library/][1]\n - [https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.v471kaepx][2]\n\n\n [1]: http://machinelearningmastery.com/object-recognition-convolutional-neural-networks-keras-deep-learning-library/\n [2]: https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.v471kaepx",
"metadata": {}
},
{
"cell_type": "markdown",
"source": "## Goals\nThis Notebook is posted to give new users a start on using Keras in this competition. ",
"metadata": {}
},
{
"cell_type": "markdown",
"source": "## Results\nThis CNN will predict output with high level of accuracy, BUT: all outputs for predictions will be either 1.0 or 0.0\n\nBecause of the way this Kaggle is scored, the incorrect data points will lead to a large loss. ",
"metadata": {}
},
{
"cell_type": "markdown",
"source": "## Next steps\nNeed to find a network model/architecture that will distribute its predictions over the full set of classes, not simply a 0/1 binary prediction. ",
"metadata": {}
},
{
"cell_type": "code",
"source": "# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# imports needed for CNN\nimport csv\nimport cv2\nimport os, glob\nimport pandas as pd\nimport numpy as np\nimport sklearn\nfrom sklearn.model_selection import train_test_split\nimport time\nfrom keras.datasets import cifar10\nfrom keras.models import Sequential\nfrom keras.layers import Dense\nfrom keras.layers import Dropout\nfrom keras.layers import Flatten\nfrom keras.constraints import maxnorm\nfrom keras.optimizers import SGD\nfrom keras.layers.convolutional import Convolution2D\nfrom keras.layers.convolutional import MaxPooling2D\nfrom keras.utils import np_utils\nfrom matplotlib import pyplot as plt\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list the files in the input directory\n\nfrom subprocess import check_output\nprint(check_output([\"ls\", \"../input\"]).decode(\"utf8\"))\n\n# Load the data\ndef load_data(data_dir):\n \"\"\"\n From: https://medium.com/@waleedka/traffic-sign-recognition-with-tensorflow-629dffc391a6#.v471kaepx\n \"\"\"\n # Get all subdirectories of data_dir. Each represents a label.\n directories = [d for d in os.listdir(data_dir)\n if os.path.isdir(os.path.join(data_dir, d))]\n # Loop through the label directories and collect the data in\n # two lists, labels and images.\n labels = []\n images = []\n\n category = 0\n for d in directories:\n label_dir = os.path.join(data_dir, d)\n file_names = [os.path.join(label_dir, f)\n for f in os.listdir(label_dir)\n if f.endswith(\".jpg\")]\n \n # adding an early stop for sake of speed\n stop = 0\n for f in file_names:\n img = cv2.imread(f)\n imresize = cv2.resize(img, (200, 125))\n #plt.imshow(imresize)\n images.append(imresize)\n labels.append(category)\n # remove this to use full data set\n if stop > 30:\n break\n stop += 1\n # end early stop\n \n category += 1\n\n return images, labels\n\ndata_dir = \"../input\"\nimages, labels = load_data(data_dir)\n\n# confirm that we have the data\nprint(images[0:10])\nprint(labels[0:10])",
"execution_count": null,
"outputs": [],
"metadata": {}
},
{
"cell_type": "markdown",
"source": "Cross validate the data, so we can use a test set to check accuracy, before submitting.",
"metadata": {}
},
{
"cell_type": "code",
"source": "def cross_validate(Xs, ys):\n X_train, X_test, y_train, y_test = train_test_split(\n Xs, ys, test_size=0.2, random_state=0)\n return X_train, X_test, y_train, y_test\n\nX_train, X_test, y_train, y_test = cross_validate(images, labels)\n\n# confirm we got our data\nprint(y_test[0:10])",
"execution_count": null,
"outputs": [],
"metadata": {}
},
{
"cell_type": "markdown",
"source": "Normalize the data and hot encode outputs",
"metadata": {}
},
{
"cell_type": "code",
"source": " # normalize inputs from 0-255 and 0.0-1.0\n X_train = np.array(X_train).astype('float32')\n X_test = np.array(X_test).astype('float32')\n X_train = X_train / 255.0\n X_test = X_test / 255.0\n\n # one hot encode outputs\n y_train = np.array(y_train)\n y_test = np.array(y_test)\n y_train = np_utils.to_categorical(y_train)\n y_test = np_utils.to_categorical(y_test)\n num_classes = y_test.shape[1]\n print(\"Data normalized and hot encoded.\")",
"execution_count": null,
"outputs": [],
"metadata": {}
},
{
"cell_type": "markdown",
"source": "### The code below creates and fits the CNN. It will take a while to load, even with 2 epochs.\nPlease ensure the code below runs, before testing the final section, which will save your file. ",
"metadata": {}
},
{
"cell_type": "markdown",
"source": "Create our CNN Model",
"metadata": {}
},
{
"cell_type": "code",
"source": "def createCNNModel(num_classes):\n \"\"\" Adapted from: # http://machinelearningmastery.com/object-recognition-convolutional-neural-networks-keras-deep-learning-library/\n# \"\"\"\n # Create the model\n model = Sequential()\n model.add(Convolution2D(32, 3, 3, input_shape=(125, 200, 3), border_mode='same', activation='relu', W_constraint=maxnorm(3)))\n model.add(Dropout(0.2))\n model.add(Convolution2D(32, 3, 3, activation='relu', border_mode='same', W_constraint=maxnorm(3)))\n model.add(MaxPooling2D(pool_size=(2, 2)))\n model.add(Flatten())\n model.add(Dense(512, activation='relu', W_constraint=maxnorm(3)))\n model.add(Dropout(0.5))\n model.add(Dense(num_classes, activation='softmax'))\n # Compile model\n epochs = 3 # >>> should be 25+\n lrate = 0.01\n decay = lrate/epochs\n sgd = SGD(lr=lrate, momentum=0.9, decay=decay, nesterov=False)\n model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])\n print(model.summary())\n return model, epochs\n\n# create our CNN model\nmodel, epochs = createCNNModel(num_classes)\nprint(\"CNN Model created.\")",
"execution_count": null,
"outputs": [],
"metadata": {}
},
{
"cell_type": "markdown",
"source": "Fit and run the model",
"metadata": {}
},
{
"cell_type": "code",
"source": " # fit and run our model\n seed = 7\n np.random.seed(seed)\n model.fit(X_train, y_train, validation_data=(X_test, y_test), nb_epoch=epochs, batch_size=64)\n # Final evaluation of the model\n scores = model.evaluate(X_test, y_test, verbose=0)\n print(\"Accuracy: %.2f%%\" % (scores[1]*100))\n\n print(\"done\")",
"execution_count": null,
"outputs": [],
"metadata": {}
},
{
"cell_type": "markdown",
"source": "Save output for upload \n\n**Make sure to run the above code first, otherwise the model will not be defined**",
"metadata": {}
},
{
"cell_type": "code",
"source": " from os import listdir\n from os.path import isfile, join\n\n prediction_output_list = [] # list of lists, containing logistic regression for each file\n fnames = [f for f in listdir(\"../input/test_stg1/\") if isfile(join(\"../input/test_stg1/\", f))]\n print(\"Testing File Names:\")\n print(fnames)\n\n # early stoppage...\n # only do 10\n i = 0\n for f in fnames:\n file_name = \"../input/test_stg1/\" + f\n print(\"---Evaluating File at: \" + file_name)\n img = cv2.imread(file_name) \n imresize = cv2.resize(img, (200, 125)) # resize so we're always comparing same-sized images\n imlist = np.array([imresize])\n print(\"Neural Net Prediction:\")\n cnn_prediction = model.predict_proba(imlist)\n print(cnn_prediction)\n\n # format list for csv output\n csv_output_list = []\n csv_output_list.append(f)\n for elem in cnn_prediction:\n for value in elem:\n csv_output_list.append(value)\n\n # append filename to make sure we have right format to write to csv\n print(\"CSV Output List Formatted:\")\n print(csv_output_list)\n\n\n # and append this file to the output_list (of lists)\n prediction_output_list.append(csv_output_list)\n \n ############## STOP EARLY TO SAVE TIME #################\n if i > 10:\n break\n i += 1\n ##### REMOVE TO RUN AGAINST FULL TEST SET ########\n\n # Write to csv\n \"\"\"\n # Commented out for Kaggle, but you can use this to write to a CSV on your own computer.\n try:\n with open(\"cnn_predictions.csv\", \"wb\") as f:\n writer = csv.writer(f)\n headers = ['image', 'ALB', 'BET', 'DOL', 'LAG', 'NoF', 'OTHER', 'SHARK', 'YFT']\n writer.writerow(headers)\n writer.writerows(prediction_output_list)\n finally:\n f.close()\n \"\"\"\n\n print(\"done\")",
"execution_count": null,
"outputs": [],
"metadata": {}
}
]
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment