This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#luarocks install loadcaffe | |
#luarocks install hdf5 | |
#git clone https://github.com/karpathy/neuraltalk2; cd neuraltalk2 | |
#wget http://cs.stanford.edu/people/karpathy/neuraltalk2/checkpoint_v1.zip | |
#unzip checkpoint_v1.zip -d data | |
#mkdir samples; cd samples | |
#wget https://s3-eu-west-1.amazonaws.com/dl-npl/img.zip | |
#unzip img.zip |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
from skimage import exposure | |
def grayscale_exposure_equalize(batch_x_y): | |
"""Processes a batch with images by grayscaling, normalization and | |
histogram equalization. | |
Args: | |
batch_x_y: a single batch of data containing a numpy array of images | |
and a list of corresponding labels. | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import multiprocessing | |
import ipyparallel as ipp | |
import numpy as np | |
def preprocess_equalize(X, y, bins=256, cpu=multiprocessing.cpu_count()): | |
""" A simplified version of a function which manages multiprocessing logic. | |
This function always grayscales input images, though it can be generalized | |
to apply any arbitrary function to batches. | |
Args: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import numpy as np | |
from skimage import transform | |
from skimage.transform import warp, AffineTransform | |
def rotate_90_deg(X): | |
X_aug = np.zeros_like(X) | |
for i,img in enumerate(X): | |
X_aug[i] = transform.rotate(img, 270.0) | |
return X_aug |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
label_class | label_name | rotate_90_deg | rotate_180_deg | rotate_270_deg | flip_horiz | flip_vert | |
---|---|---|---|---|---|---|---|
13 | Yield | 13 | |||||
14 | Stop | ||||||
15 | No vehicles | 15 | 15 | 15 | 15 | 15 | |
16 | Vehicles over 3.5 metric tons prohibited | ||||||
17 | No entry | 17 | 17 | 17 | |||
18 | General caution | 18 | |||||
19 | Dangerous curve to the left | 20 | |||||
20 | Dangerous curve to the right | 19 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import pandas as pd | |
# Generate an augmented dataset using a transform table | |
augmentation_table = pd.read_csv('augmentation_table.csv', index_col='label_class') | |
augmentation_table.drop('label_name', axis=1, inplace=True) | |
augmentation_table.dropna(axis=0, how='all', inplace=True) | |
# Collect all global functions in global namespace | |
namespace = __import__(__name__) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import numpy as np | |
def flips_rotations_augmentation(X, y): | |
"""A pipeline for applying augmentation functions listed in `augmentation_table` | |
to a numpy array with images X. | |
""" | |
# Initializing empty arrays to accumulate intermediate results of augmentation | |
X_out, y_out = np.empty([0] + list(X.shape[1:]), dtype=np.float32), np.empty([0]) | |
# Cycling through all label classes and applying all available transformations |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
def augmented_batch_generator(X, y, batch_size, rotations=True, affine=True, | |
shear_angle=0.0, scale_margins=[0.8, 1.5], p=0.35): | |
"""Augmented batch generator. Splits the dataset into batches and augments each | |
batch independently. | |
Args: | |
X: numpy array with images. | |
y: list of labels. | |
batch_size: the size of the output batch. | |
rotations: whether to apply `flips_rotations_augmentation` function to dataset. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
label_class | label_name | rotate_90_deg | rotate_180_deg | rotate_270_deg | flip_horiz | flip_vert | |
---|---|---|---|---|---|---|---|
0 | Speed limit (20km/h) | ||||||
1 | Speed limit (30km/h) | ||||||
2 | Speed limit (50km/h) | ||||||
3 | Speed limit (60km/h) | ||||||
4 | Speed limit (70km/h) | ||||||
5 | Speed limit (80km/h) | ||||||
6 | End of speed limit (80km/h) | ||||||
7 | Speed limit (100km/h) | ||||||
8 | Speed limit (120km/h) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
echo | |
echo "This installation sets up a P2 instance on AWS (Ubuntu 14.04 only), installs all necessary packages" | |
echo "which are required for CUDA, TensorFlow, Keras, Caffe and Torch." | |
echo | |
# Install dependencies | |
sudo apt-get update | |
sudo apt-get --assume-yes upgrade | |
sudo apt-get install -y build-essential git python-pip libfreetype6-dev libxft-dev libncurses-dev libopenblas-dev gfortran python-matplotlib libblas-dev liblapack-dev libatlas-base-dev python-dev python-pydot linux-headers-generic linux-image-extra-virtual unzip python-numpy swig python-pandas python-sklearn zip | |
sudo pip install -U pip |