Difference of stuctural similarity using Tensorflow and keras. Works ONLY on tf >= 0.11

DSSIM.py

import keras.backend as K
import tensorflow as tf
class Model:
  def __init__(self,batch_size):
    self.batch_size = batch_size
    
    def loss_DSSIS_tf11(self, y_true, y_pred):
        """Need tf0.11rc to work"""
        y_true = tf.reshape(y_true, [self.batch_size] + get_shape(y_pred)[1:])
        y_pred = tf.reshape(y_pred, [self.batch_size] + get_shape(y_pred)[1:])
        y_true = tf.transpose(y_true, [0, 2, 3, 1])
        y_pred = tf.transpose(y_pred, [0, 2, 3, 1])
        patches_true = tf.extract_image_patches(y_true, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
        patches_pred = tf.extract_image_patches(y_pred, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")

        u_true = K.mean(patches_true, axis=3)
        u_pred = K.mean(patches_pred, axis=3)
        var_true = K.var(patches_true, axis=3)
        var_pred = K.var(patches_pred, axis=3)
        std_true = K.sqrt(var_true)
        std_pred = K.sqrt(var_pred)
        c1 = 0.01 ** 2
        c2 = 0.03 ** 2
        ssim = (2 * u_true * u_pred + c1) * (2 * std_pred * std_true + c2)
        denom = (u_true ** 2 + u_pred ** 2 + c1) * (var_pred + var_true + c2)
        ssim /= denom
        ssim = tf.select(tf.is_nan(ssim), K.zeros_like(ssim), ssim)
        return K.mean(((1.0 - ssim) / 2))

I am also getting the same error. Can you explain how to use this loss while training ?

@ckyrkou @harryvineeth : you could try using the loss as standalone method. Something like that:

def make_loss(batch_size):
    def loss(y_true, y_pred):
        # batch_size can be accessed
        return 1  # computation of SSIM/DSSIM 
    return loss

minor note:

y_true = tf.reshape(y_true, [self.batch_size] + get_shape(y_pred)[1:])
y_pred = tf.reshape(y_pred, [self.batch_size] + get_shape(y_pred)[1:])

Shouldnt the first y_pred be named y_true? (I dont think it makes much of a difference though...

Anyhow, here is a version that works with my current version of tensorflow (1.11)

def ssim_metric(y_true, y_pred):
    # source: https://gist.github.com/Dref360/a48feaecfdb9e0609c6a02590fd1f91b

    y_true = tf.expand_dims(y_true, -1)
    y_pred = tf.expand_dims(y_pred, -1)
    y_true = tf.transpose(y_true, [0, 2, 3, 1])
    y_pred = tf.transpose(y_pred, [0, 2, 3, 1])
    patches_true = tf.extract_image_patches(y_true, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")
    patches_pred = tf.extract_image_patches(y_pred, [1, 5, 5, 1], [1, 2, 2, 1], [1, 1, 1, 1], "SAME")

    u_true = K.mean(patches_true, axis=3)
    u_pred = K.mean(patches_pred, axis=3)
    var_true = K.var(patches_true, axis=3)
    var_pred = K.var(patches_pred, axis=3)
    std_true = K.sqrt(var_true)
    std_pred = K.sqrt(var_pred)
    c1 = 0.01 ** 2
    c2 = 0.03 ** 2
    ssim = (2 * u_true * u_pred + c1) * (2 * std_pred * std_true + c2)
    denom = (u_true ** 2 + u_pred ** 2 + c1) * (var_pred + var_true + c2)
    ssim /= denom
    ssim = tf.where(tf.is_nan(ssim), K.zeros_like(ssim), ssim)
    return ssim)

I had to manually add the last channel via tf.expand_dims, since I had 2d arrays as a result

Hi @zwep, I'm a beginner at deep Learning and I have to use "SSIM Loss function", cloud you please help me?

Hi @Dref360, I'm a beginner at deep Learning and I have to use "SSIM Loss function", cloud you please help me?

Hi @Dref360, I don't understand this part - (2 * std_pred * std_true + c2).
Aren't we suppose to have covariance(u_true,u_pred) and not this. I am not able to understand the difference between covariance(x,y) and std(x)*std(y) in your code.

Hi! In this case, we assume the independance between both images so we get that cov(x,y) = -(sigma_x * sigma-y). In my work I made this assumption, I don'ty know if it's the right one. Cheers, Fred

…

________________________________ De : Aaradhya Saxena <notifications@github.com> Envoyé : 10 juillet 2019 06:40:34 À : Dref360 Cc : Frédéric Branchaud-Charron; Mention Objet : Re: Dref360/DSSIM.py Hi @Dref360<https://github.com/Dref360>, I don't understand this part - (2 * std_pred * std_true + c2). Aren't we suppose to have covariance(u_true,u_pred) and not this. I am not able to understand the difference between covariance(x,y) and std(x)*std(y) in your code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://gist.github.com/a48feaecfdb9e0609c6a02590fd1f91b?email_source=notifications&email_token=ACEPRIRU2SCKGN4H73WPVOTP6W4CFA5CNFSM4H7N7DB2YY3PNVWWK3TUL52HS4DFVNDWS43UINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAFVBZQ#gistcomment-2966424>, or mute the thread<https://github.com/notifications/unsubscribe-auth/ACEPRIUBBL2PRR7P2R5U4R3P6W4CFANCNFSM4H7N7DBQ>.

May you provide a license for this code, please?

You can probably use https://www.tensorflow.org/api_docs/python/tf/image/ssim
In any case, this has been merged to keras-contrib a while ago.

Ok, thank you very much! Il giorno mar 15 ott 2019 alle ore 16:17 Frédéric Branchaud-Charron < notifications@github.com> ha scritto:

…

You can probably use https://www.tensorflow.org/api_docs/python/tf/image/ssim In any case, this has been merged to keras-contrib a while ago. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <https://gist.github.com/a48feaecfdb9e0609c6a02590fd1f91b?email_source=notifications&email_token=ADJ3F5PP7T6PAQTUOZU2HF3QOXGHVA5CNFSM4H7N7DB2YY3PNVWWK3TUL52HS4DFVNDWS43UINXW23LFNZ2KUY3PNVWWK3TUL5UWJTQAF2QOU#gistcomment-3055850>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ADJ3F5JCXFP7MWEEH3ZMSLDQOXGHVANCNFSM4H7N7DBQ> .