Notes on SOTA Super-resolution

Overall notes.md

Things to try for neural networks:

General adversary network:

https://arxiv.org/pdf/1609.04802v5.pdf
Tensorflow: models

Idea:

Current models are good at maximising MSE(mean squared error) of the image, however, this metric leads to more smoothing out of the noise and thus leads to loss of detail. A new method of using the 'feature maps' of the VGG(Pretrained loss function CNN) Their GAN architecture works by getting a model to train try and estimate original image and another one trying to descriminate between the the estimate and the original. These two networks train together, the generator being scored on a combination of the VGG mentioned above and the ability of the descriminator to tell the difference.

Enhanced adversarial network:

https://arxiv.org/pdf/1809.00219v2.pdf
https://github.com/xinntao/ESRGAN

Ideas:

Relativisic discriminator: Normal descriminator gets too good at detetcting what is true and what is false and overfits, this leads to not much use out of it. To fix this introduce a 'bias' such that discriminator is less sure of real images.

Image Reconstruction Filter flow:

https://arxiv.org/pdf/1811.11482v1.pdf
https://github.com/aimerykong/predictive-filter-flow

Idea:

Instead of link low res image to high res image, link low res image to a linear filter for itself that when applied createsthe high res image, the advantage being that its easier to make local changes as compared to a fully connected pixel to pixel model and its more versatile than a convelution(filters around different points can have different values).

Second-order 'Attention Network'

http://openaccess.thecvf.com/content_CVPR_2019/papers/Dai_Second-Order_Attention_Network_for_Single_Image_Super-Resolution_CVPR_2019_paper.pdf
https://github.com/daitao/SAN

Idea:

The human mind is incredibly good at focusing the attention of our eyes on one specific part of the full spectrum of information that is in front of us(eg, human faces), it could be a good idea to replicate this in convelutional neural networks. To do this first, extract low level features of a network using a convelutional layer, then combine this layer with a progressively higher level features using a new LSRAG module which is fitted with a second order feature extractor thus getting more info from the input, all of this combined makes the network focus more on the features of the image rather than the values themselves thus getting more performance.

Multy Feedback Networks:

https://arxiv.org/pdf/1907.04253v2.pdf
https://github.com/liqilei/GMFN

Idea:

Feed output back into input recursibley creating a chane of CNN's thus making each CNN only do a small part of the full task making them easier to train and more effective