jdsgomes

Speed Improvements and Compression for Deep Learning

Notes

• Quantized Convolutional Neural Networks for Mobile Devices CVPR 2016
• Deep SimNets CVPR 2016
• Quantized Convolutional Neural Networks for Mobile Devices (2016)
  • Quantization is applied to both convolutional and fully connected layers. This method has the advantage of accelerating the convolutional layers runtime, which is very important given that those are the most computationally expensive layers in CNNs. The disadvantage of this method is that it leads to a small loss in accuracy.
• SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size (2016)
• Use 1x1 convolutions to reduce the number of convolution maps. I.e. if we have 10 traditional convolution filters this will generate 10 feature maps. If then we have N convolutions (where N < 10) the res