| import json | |
| import os | |
| import numpy as np | |
| from sklearn.metrics import accuracy_score | |
| from sklearn.svm import LinearSVC | |
| # File Paths | |
| DATA_FOLDER = '' # Fill in | |
| TRAIN_DATASET_FP = os.path.join(DATA_FOLDER, 'train2018.json'); assert os.path.exists(TRAIN_DATASET_FP), "Train json file not found" |
Installing TensorFlow r1.1 on an AWS P2 instance.
Following this with a few modifications
Built using Ubuntu 16.04 with p2.xlarge instance and 25GB of standard ssd storage.
# Update and upgrade installed packages
sudo apt-get update
sudo apt-get upgrade
Installing TensorFlow r1.1 on an AWS C4 instance.
Following this with a few modifications
Built using Ubuntu 16.04 with c4.2xlarge instance and 25GB of standard ssd storage.
# Update and upgrade installed packages
sudo apt-get update
sudo apt-get upgrade
| export DATASET_DIR=/media/drive2/tensorflow_datasets/cub/with_200_val_split | |
| export EXPERIMENT_DIR=/media/drive2/tensorflow_experiments/ebird/cub_image_experiment | |
| export IMAGENET_PRETRAINED_MODEL=/media/drive3/tensorflow_models/inception_v3.ckpt | |
| # Visualize the inputs to the network | |
| CUDA_VISIBLE_DEVICES=1 python visualize_train_inputs.py \ | |
| --tfrecords $DATASET_DIR/train* \ | |
| --config $EXPERIMENT_DIR/config_train.yaml \ | |
| --text_labels |
Instructions for installing TensorFlow Serving on Ubuntu 14.04. I am following the instuctions from here.
Installation instructions can be found here
If you have a previous version of bazel, and you are trying to do a fresh install then you should remove your old version of bazel. If you installed it through apt, then you can do sudo apt-get purge bazel. If you installed it from source, then you probably have a ~/bin directory with the bazel command, which you should delete, and you probably have a ~/.bazel directory that you should delete. Also check your ~/.bashrc file for any links to ~/.bazel.
add-apt-repository command, which you can get by doing sudo apt-get install software-properties-common
| # Testing specific configuration | |
| RANDOM_SEED : 1.0 | |
| SESSION_CONFIG : { | |
| LOG_DEVICE_PLACEMENT : false, | |
| PER_PROCESS_GPU_MEMORY_FRACTION : 0.9 | |
| } | |
| ################################################# | |
| # Metrics |
| # Training specific configuration | |
| RANDOM_SEED : 1.0 | |
| SESSION_CONFIG : { | |
| LOG_DEVICE_PLACEMENT : false, | |
| PER_PROCESS_GPU_MEMORY_FRACTION : 0.94 | |
| } | |
| ################################################# | |
| # Dataset Info |
| import cPickle as pickle | |
| import os | |
| dataset_dir = '.' | |
| train_file = os.path.join(dataset_dir, "train_data.pkl") | |
| with open(train_file) as f: | |
| train_data = pickle.load(f) | |
| image_dir = "/home/gvanhorn/datasets/inaturalist/images" | |
| train_images = [] |
Assuming you have a file urls.txt that has, for each row, the name of file to save and the url to fetch, space separated. You can then use the following to download the urls.
parallel -j8 --colsep " " "wget -q -O {1} {2}" < urls.txt
parallel --eta -j4 --colsep " " "wget -q -N -t 1 -T 10 -O {1} {2}" < urls.txt
-q for quiet
| import os | |
| import random | |
| import sys | |
| from collections import Counter | |
| def format_labels(image_labels): | |
| """ | |
| Convert the image labels to be integers between [0, num classes) | |
| Returns : |