GSoC 2017 - Improve Background Subtraction with Aggregated Saliency

GSoC2017_Improve_Background_Subtraction_with_Aggregated_Saliency.md

Improve Background Subtraction with Aggregated Saliency

Contributor: Shengxin Qian | Mentor: Antonella Cascitelli

1. Overview

This project contains the implementation of three saliency algorithms:

• DeepGaze1 [1] Image based saliency

  • This is a supervised learning algorithm based on the linear combination of AlexNet 5th convolution layers.
  • This algorithm takes advantage of the high level features generated by those pretrained Deep Neural Network.
  • The training of this algorithm is slow but it is fast when estimating saliency.
  • This algorithm can achieve 0.84 AUC on MIT Saliency Benchmark which is one of the best score.
  • This algorithm can be retrained with cutting edge Deep Neural Network and achieve much better performance.

• Discriminant Saliency [2] Video based saliency

  • This is an unsupervised learning algorithm based on dynamic texture model and “center surround” local feature contrast.
  • This algorithm exhibits substantial robustness, performing well in the presence of camera motion, variable foreground object scales, and low imaging quality
  • The drawback of this algorithm is its the processing speed. Depending on the output resolution, each frame takes from several seconds to more than a hundred seconds. However, it has potential to be highly scalable if you can slightly modify it to parallel version because the processing of each sliding window is independent.

• Background Contrast [3] Image based saliency

  • This is an unsupervised learning algorithm based on superpixels' boundary connectivity and region feature contrast.
  • This algorithm is able to estimate the probability of being background objects and be effective on those images that saliency objects occupy certain amount of space.
  • The downside of this algorithm is it can not handle those images when high level cues is critical or the scene is complex.
  • This module offers the opportunity of optimizing the result of other saliency algorithms and segmenting those saliency objects.

List of commits during GSoC 2017

• All commits can be found here: https://github.com/opencv/opencv_contrib/pull/1252/commits

• All codes can be found here: https://github.com/opencv/opencv_contrib/pull/1252/files

Reference

1. Kümmerer, Matthias, Lucas Theis, and Matthias Bethge. "Deep gaze i: Boosting saliency prediction with feature maps trained on imagenet." arXiv preprint arXiv:1411.1045 (2014).
2. Mahadevan, Vijay, and Nuno Vasconcelos. "Spatiotemporal saliency in dynamic scenes." IEEE transactions on pattern analysis and machine intelligence 32.1 (2010): 171-177.
3. Zhu, Wangjiang, et al. "Saliency optimization from robust background detection." Proceedings of the IEEE conference on computer vision and pattern recognition . 2014.