Skip to content

Instantly share code, notes, and snippets.

@Dref360
Last active Aug 3, 2020
Embed
What would you like to do?
Difference of stuctural similarity using Tensorflow and keras. Works ONLY on tf >= 0.11
import keras.backend as K
import tensorflow as tf
class Model:
def __init__(self,batch_size):
self.batch_size = batch_size
def loss_DSSIS_tf11(self, y_true, y_pred):
"""Need tf0.11rc to work"""
y_true = tf.reshape(y_true, [self.batch_size] + get_shape(y_pred)[1:])
y_pred = tf.reshape(y_pred, [self.batch_size] + get_shape(y_pred)[1:])
y_true = tf.transpose(y_true, [0, 2, 3, 1])
y_pred = tf.transpose(y_pred, [0, 2, 3, 1])
patches_true = tf.extract_image_patches(y_true, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
patches_pred = tf.extract_image_patches(y_pred, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
u_true = K.mean(patches_true, axis=3)
u_pred = K.mean(patches_pred, axis=3)
var_true = K.var(patches_true, axis=3)
var_pred = K.var(patches_pred, axis=3)
std_true = K.sqrt(var_true)
std_pred = K.sqrt(var_pred)
c1 = 0.01 ** 2
c2 = 0.03 ** 2
ssim = (2 * u_true * u_pred + c1) * (2 * std_pred * std_true + c2)
denom = (u_true ** 2 + u_pred ** 2 + c1) * (var_pred + var_true + c2)
ssim /= denom
ssim = tf.select(tf.is_nan(ssim), K.zeros_like(ssim), ssim)
return K.mean(((1.0 - ssim) / 2))
@ckyrkou
Copy link

ckyrkou commented Nov 24, 2017

Can you also provide an example on how to actually use this to train a model in Keras? Thanks.
When I tried to use it like this:

model.compile(loss=loss_DSSIS_tf11, optimizer=adam)

I get the error:

TypeError: loss_DSSIS_tf11() missing 1 required positional argument: 'y_pred'

@harryvineeth
Copy link

harryvineeth commented Mar 6, 2018

I am also getting the same error. Can you explain how to use this loss while training ?

@rposer
Copy link

rposer commented Jul 25, 2018

@ckyrkou @harryvineeth : you could try using the loss as standalone method. Something like that:

def make_loss(batch_size):
    def loss(y_true, y_pred):
        # batch_size can be accessed
        return 1  # computation of SSIM/DSSIM 
    return loss

@zwep
Copy link

zwep commented Nov 26, 2018

minor note:

y_true = tf.reshape(y_true, [self.batch_size] + get_shape(y_pred)[1:])
y_pred = tf.reshape(y_pred, [self.batch_size] + get_shape(y_pred)[1:])

Shouldnt the first y_pred be named y_true? (I dont think it makes much of a difference though...

@zwep
Copy link

zwep commented Nov 26, 2018

Anyhow, here is a version that works with my current version of tensorflow (1.11)


def ssim_metric(y_true, y_pred):
    # source: https://gist.github.com/Dref360/a48feaecfdb9e0609c6a02590fd1f91b

    y_true = tf.expand_dims(y_true, -1)
    y_pred = tf.expand_dims(y_pred, -1)
    y_true = tf.transpose(y_true, [0, 2, 3, 1])
    y_pred = tf.transpose(y_pred, [0, 2, 3, 1])
    patches_true = tf.extract_image_patches(y_true, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
    patches_pred = tf.extract_image_patches(y_pred, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")

    u_true = K.mean(patches_true, axis=3)
    u_pred = K.mean(patches_pred, axis=3)
    var_true = K.var(patches_true, axis=3)
    var_pred = K.var(patches_pred, axis=3)
    std_true = K.sqrt(var_true)
    std_pred = K.sqrt(var_pred)
    c1 = 0.01 ** 2
    c2 = 0.03 ** 2
    ssim = (2 * u_true * u_pred + c1) * (2 * std_pred * std_true + c2)
    denom = (u_true ** 2 + u_pred ** 2 + c1) * (var_pred + var_true + c2)
    ssim /= denom
    ssim = tf.where(tf.is_nan(ssim), K.zeros_like(ssim), ssim)
    return ssim)

I had to manually add the last channel via tf.expand_dims, since I had 2d arrays as a result

@alirezaIzd
Copy link

alirezaIzd commented Jan 7, 2019

Hi @zwep, I'm a beginner at deep Learning and I have to use "SSIM Loss function", cloud you please help me?

@alirezaIzd
Copy link

alirezaIzd commented Jan 7, 2019

Hi @Dref360, I'm a beginner at deep Learning and I have to use "SSIM Loss function", cloud you please help me?

@AaradhyaSaxena
Copy link

AaradhyaSaxena commented Jul 10, 2019

Hi @Dref360, I don't understand this part - (2 * std_pred * std_true + c2).
Aren't we suppose to have covariance(u_true,u_pred) and not this. I am not able to understand the difference between covariance(x,y) and std(x)*std(y) in your code.

@Dref360
Copy link
Author

Dref360 commented Jul 10, 2019

@Aspie96
Copy link

Aspie96 commented Oct 15, 2019

May you provide a license for this code, please?

@Dref360
Copy link
Author

Dref360 commented Oct 15, 2019

You can probably use https://www.tensorflow.org/api_docs/python/tf/image/ssim
In any case, this has been merged to keras-contrib a while ago.

@Aspie96
Copy link

Aspie96 commented Oct 15, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment