gvanhorn38

You can install imagemagick with:

sudo apt-get install imagemagick

We first need to identify potential problems:

#!/bin/bash

gvanhorn38

import tensorflow as tf

def _float_feature(value):
  return tf.train.Feature(float_list=tf.train.FloatList(value=value))

def _int64_feature(value):
  return tf.train.Feature(int64_list=tf.train.Int64List(value=value))

def _bytes_feature(value):
  return tf.train.Feature(bytes_list=tf.train.BytesList(value=value))

gvanhorn38

import os
import sys

def format_labels(image_labels):
  """
  Convert the image labels to be integers between [0, num classes)
  
  Returns : 
    condensed_image_labels = { image_id : new_label}
    new_id_to_original_id_map = {new_label : original_label}

gvanhorn38

We had some issues when we tried to download the CUDA Toolkit .deb file and use the apt-get package manager to install CUDA Toolkit on Ubuntu 15.04. So instead we just manually installed the CUDA Toolkit .run file.

We used these resources to piece together the right steps for doing this. http://developer.download.nvidia.com/compute/cuda/7.5/Prod/docs/sidebar/CUDA_Installation_Guide_Linux.pdf http://www.allaboutlinux.eu/remove-nouveau-and-install-nvidia-driver-in-ubuntu-15-04/2/ https://askubuntu.com/questions/16371/how-do-i-disable-x-at-boot-time-so-that-the-system-boots-in-text-mode

Remove existing nvidia stuff:

sudo apt-get remove nvidia*

gvanhorn38

import os
import random
import sys
from collections import Counter

def format_labels(image_labels):
  """
  Convert the image labels to be integers between [0, num classes)
  
  Returns : 

gvanhorn38

Assuming you have a file urls.txt that has, for each row, the name of file to save and the url to fetch, space separated. You can then use the following to download the urls.

parallel -j8 --colsep " " "wget -q -O {1} {2}" < urls.txt

parallel --eta -j4 --colsep " " "wget -q -N -t 1 -T 10 -O {1} {2}" < urls.txt

-q for quiet

gvanhorn38

import cPickle as pickle
import os

dataset_dir = '.'
train_file = os.path.join(dataset_dir, "train_data.pkl")
with open(train_file) as f:
  train_data = pickle.load(f)

image_dir = "/home/gvanhorn/datasets/inaturalist/images"
train_images = []

gvanhorn38

# Training specific configuration
RANDOM_SEED : 1.0

SESSION_CONFIG : {
  LOG_DEVICE_PLACEMENT : false,
  PER_PROCESS_GPU_MEMORY_FRACTION : 0.94
}

#################################################
# Dataset Info

gvanhorn38

# Testing specific configuration
RANDOM_SEED : 1.0

SESSION_CONFIG : {
  LOG_DEVICE_PLACEMENT : false,
  PER_PROCESS_GPU_MEMORY_FRACTION : 0.9
}
#################################################
# Metrics

gvanhorn38

Instructions for installing TensorFlow Serving on Ubuntu 14.04. I am following the instuctions from here.

Install Bazel

Installation instructions can be found here

If you have a previous version of bazel, and you are trying to do a fresh install then you should remove your old version of bazel. If you installed it through apt, then you can do sudo apt-get purge bazel. If you installed it from source, then you probably have a ~/bin directory with the bazel command, which you should delete, and you probably have a ~/.bazel directory that you should delete. Also check your ~/.bashrc file for any links to ~/.bazel.

1. Install JDK 8 You'll need the add-apt-repository command, which you can get by doing sudo apt-get install software-properties-common