huyhoang17

Module autoreload

To auto-reload modules in jupyter notebook (so that changes in files *.py doesn't require manual reloading):

# for auto-reloading external modules
# see http://stackoverflow.com/questions/1907993/autoreload-of-modules-in-ipython
%load_ext autoreload
%autoreload 2

huyhoang17

#!/bin/bash
## This gist contains step by step instructions to install cuda v9.0 and cudnn 7.2 in ubuntu 18.04

### steps ####
# verify the system has a cuda-capable gpu
# download and install the nvidia cuda toolkit and cudnn
# setup environmental variables
# verify the installation
###

huyhoang17

huyhoang17

Troubleshooting Convolutional Neural Networks

Intro

This is a list of hacks gathered primarily from prior experiences as well as online sources (most notably Stanford's CS231n course notes) on how to troubleshoot the performance of a convolutional neural network . We will focus mainly on supervised learning using deep neural networks. While this guide assumes the user is coding in Python3.6 using tensorflow (TF), it can still be helpful as a language agnostic guide.

Suppose we are given a convolutional neural network to train and evaluate and assume the evaluation results are worse than expected. The following are steps to troubleshoot and potentially improve performance. The first section corresponds to must-do's and generally good practices before you start troubleshooting. Every subsequent section header corresponds to a problem and the section is devoted to solving it. The sections are ordered to reflect "more common" issues first and under each header the "most-eas

huyhoang17

Movies Recommendation:

• MovieLens - Movie Recommendation Data Sets http://www.grouplens.org/node/73
• Yahoo! - Movie, Music, and Images Ratings Data Sets http://webscope.sandbox.yahoo.com/catalog.php?datatype=r
• Jester - Movie Ratings Data Sets (Collaborative Filtering Dataset) http://www.ieor.berkeley.edu/~goldberg/jester-data/
• Cornell University - Movie-review data for use in sentiment-analysis experiments http://www.cs.cornell.edu/people/pabo/movie-review-data/

Music Recommendation:

• Last.fm - Music Recommendation Data Sets http://www.dtic.upf.edu/~ocelma/MusicRecommendationDataset/index.html

huyhoang17

"""
Simple example of building your own context manager.

Resources:

- http://preshing.com/20110920/the-python-with-statement-by-example/
- https://docs.python.org/3/library/contextlib.html
- PEP 343 -- the "with" statement: https://www.python.org/dev/peps/pep-0343/

"""

huyhoang17

##VGG16 model for Keras

This is the Keras model of the 16-layer network used by the VGG team in the ILSVRC-2014 competition.

It has been obtained by directly converting the Caffe model provived by the authors.

Details about the network architecture can be found in the following arXiv paper:

Very Deep Convolutional Networks for Large-Scale Image Recognition

K. Simonyan, A. Zisserman

huyhoang17

import numpy as np
import warnings

from itertools import cycle, izip

from sklearn.utils import gen_even_slices
from sklearn.utils import shuffle
from sklearn.base import BaseEstimator
from sklearn.base import ClassifierMixin
from sklearn.preprocessing import LabelBinarizer

huyhoang17

# taken from here: http://web.archive.org/web/20110527163743/https://svn.enthought.com/enthought/browser/sandbox/docs/coding_standard.py

""" This module is an example of the Enthought Python coding standards.

It was adapted from the Python Enhancement Proposal 8 (aka PEP 8) titled
'Style Guide for Python Code' (http://www.python.org/peps/pep-0008.html).

The first item in a module must be a documentation string (docstring).  The
first line of the docstring should be a one line summary.  If a more
detailed description is required, put an empty line before it.

huyhoang17

""" 
Play with saving .

Closest:
    https://github.com/tensorflow/tensorflow/issues/616#issuecomment-205620223
"""
import numpy as np

import tensorflow as tf
from tensorflow.python.platform import gfile