caffe-tcnn.md

Ref:

https://github.com/v-andrearczyk/caffe-TCNN

Step:

• clone onto a folder

• download the pre-trained model onto the caffe's root folder.

  • e.d. DSVM linux: /opt/caffe/models/tcnn/

• download data to /opt/caffe/data/kth-tips-2b/

Convert .jpg to lmdb, create mean image.

• copy /examples/kth-tips-2b/create_kth.sh and /examples/kth-tips-2b/make_kth_mean.sh into /opt/caffe/examples/kth-tips-2b/create_kth.sh and /opt/caffe/examples/kth-tips-2b/make_kth_mean.sh, modify paths.

• prepare according to test1.txt ... and train1.txt ...

• from /opt/caffe/,

  • run . ./examples/kth-tips-2b/create_kth.sh
  • run . ./examples/kth-tips-2b/make_kth_mean.sh
    • this will take some time

• See here for settings in create_kth.sh

To refine

• from /opt/caffe/,
  • run ./build/tools/caffe train -solver ./examples/kth-tips-2b/solver_tcnn3.prototxt -weights ./models/tcnn/tcnn3.caffemodel -gpu 0 to refine.
    • [ISSUE] Can't load the mean.binaryproto if a symbolic link is created. but work for existing example, but after customisation.
    • [FIX] Provide direct link

To train from scratch

• from /opt/caffe/,
  • run ./build/tools/caffe train -solver ./examples/kth-tips-2b/solver_tcnn3_scratch.prototxt -gpu 0 to train from scratch.

Log intermediate training loss and test accuracy

caffe train \
    -gpu 0 \
    -solver my_model/solver.prototxt \
    -weights my_model/bvlc_reference_caffenet.caffemodel 2>&1 | tee -a log/my_model.log

note: 2>&1 | tee -a log/my_model.log

Parse the log

python ./tools/extra/parse_log.py ./examples/kth-tips-2b/solver_tcnn3.log ./examples/kth-tips-2b/

Outputs:

• solver_tcnn3.log.test
• solver_tcnn3.log.train

Can then be used for plotting

Plot the log

In caffe/tools/extra there is plot_training_log.py example.

Usage:
 ./plot_log.sh chart_type[0-7] /where/to/save.png /path/to/first.log ...
Notes:
 1. Supporting multiple logs.
 2. Log file name must end with the lower-cased ".log".
Supported chart types:
 0: Test accuracy vs. Iters
 1: Test accuracy vs. Seconds
 2: Test loss vs. Iters
 3: Test loss vs. Seconds
 4: Train learning rate vs. Iters
 5: Train learning rate vs. Seconds
 6: Train loss vs. Iters
 7: Train loss vs. Seconds

Classifier to Regressor ** Need to verify **

• cp train_val_tcnn3.prototxt train_val_tcnn3_regress.prototxt

Change train_val_tcnn3_regress.prototxt as below:

...
...
...
layer {
  name: "fc8bis"
  type: "InnerProduct"
  bottom: "fc7"
  top: "fc8bis"
  param {
    lr_mult: 10
    decay_mult: 1
  }
  param {
    lr_mult: 20
    decay_mult: 0
  }
  inner_product_param {
    ###num_output: 11
    num_output: 1
    weight_filler {
      type: "gaussian"
      std: 0.01
    }
    bias_filler {
      type: "constant"
      value: 0
    }
  }
}
layer {
  name: "loss"
  ###type: "SoftmaxWithLoss"
  type: "EuclideanLoss"
  bottom: "fc8bis"
  bottom: "label"
  top: "loss"
}
######## Remove this for regression
#layer {
#  name: "accuracy"
#  type: "Accuracy"
#  bottom: "fc8bis"
#  bottom: "label"
#  top: "accuracy"
#  include {
#    phase: TEST
#  }
#}

Similarly,

• cp train_val_tcnn3.prototxt train_val_tcnn3_regress.prototxt

Change train_val_tcnn3_regress.prototxt as follows:

###net: "examples/kth-tips-2b/train_val_tcnn3.prototxt"
net: "examples/kth-tips-2b/train_val_tcnn3_regress.prototxt"
test_iter: 99 # 99x36 = 3564 (number of testing images)
test_interval: 1000
base_lr: 0.0001
lr_policy: "step"
gamma: 0.1
stepsize: 800
display: 100
max_iter: 1000
momentum: 0.9
weight_decay: 0.0005
snapshot: 10000
###snapshot_prefix: "models/tcnn/tcnn3_finetune"
snapshot_prefix: "models/tcnn/tcnn3_finetune_regress"
#solver_mode: CPU

Deploy a trained model ** Need to Verify **

cp train_val_tcnn3_regress.prototxt deploy.prototxt

Change deploy.prototxt as follows:

• Remove they type: "Data" section for both phases of TRAIN and TEST.
• Replace by `type: "Input" as follows:

name: "tcnn3CaffeNetRegress"
layer {
  name: "data"
  type: "Input"
  top: "data"
  input_param {
    shape: { dim: 1 dim: 3 dim: 227 dim: 227 } # batch size, channel, width?, height?
  }
}
...
...
...

And, remove the type: "loss" layer and `type:"accuracy" layer

...
...
...
#layer {
#  name: "loss"
#  #type: "SoftmaxWithLoss"
#  type: "EuclideanLoss"
#  bottom: "fc8bis"
#  bottom: "label"
#  top: "loss"
#}
#layer {
#  name: "accuracy"
#  type: "Accuracy"
#  bottom: "fc8bis"
#  bottom: "label"
#  top: "accuracy"
#  include {
#    phase: TEST
#  }
#}

Then, deploy from python interface.

ref:

net.forward(), caffe.Classifier(), use cv2 to prepare images

• https://mohanadkaleia.com/caffe-tutorial-build-train-and-predict/

net.forward(), caffe.Net(), use caffe.io.Transformer to prepare images

• http://gpucomputing.shef.ac.uk/education/intro_dl_sharc_dgx1/lab03/
• https://github.com/BVLC/caffe/wiki/Using-a-Trained-Network:-Deploy

net.predict(), caffe.Classifier()

• http://shengshuyang.github.io/A-step-by-step-guide-to-Caffe.html

Data Layer using MemoryData

Can only find example using MemoryData in deployment

• https://github.com/qiexing/caffe-regression/blob/master/kaggle_prototxt/fkp_deploy.prototxt and https://github.com/qiexing/caffe-regression/blob/master/kaggle_predict/predictPoint.py