Grad CAM implementation with Tensorflow 2

grad_cam.py

import cv2
import numpy as np
import tensorflow as tf

IMAGE_PATH = './cat.jpg'
LAYER_NAME = 'block5_conv3'
CAT_CLASS_INDEX = 281

img = tf.keras.preprocessing.image.load_img(IMAGE_PATH, target_size=(224, 224))
img = tf.keras.preprocessing.image.img_to_array(img)

model = tf.keras.applications.vgg16.VGG16(weights='imagenet', include_top=True)

grad_model = tf.keras.models.Model([model.inputs], [model.get_layer(LAYER_NAME).output, model.output])

with tf.GradientTape() as tape:
    conv_outputs, predictions = grad_model(np.array([img]))
    loss = predictions[:, CAT_CLASS_INDEX]

output = conv_outputs[0]
grads = tape.gradient(loss, conv_outputs)[0]

gate_f = tf.cast(output > 0, 'float32')
gate_r = tf.cast(grads > 0, 'float32')
guided_grads = tf.cast(output > 0, 'float32') * tf.cast(grads > 0, 'float32') * grads

weights = tf.reduce_mean(guided_grads, axis=(0, 1))

cam = np.ones(output.shape[0: 2], dtype = np.float32)

for i, w in enumerate(weights):
    cam += w * output[:, :, i]

cam = cv2.resize(cam.numpy(), (224, 224))
cam = np.maximum(cam, 0)
heatmap = (cam - cam.min()) / (cam.max() - cam.min())

cam = cv2.applyColorMap(np.uint8(255*heatmap), cv2.COLORMAP_JET)

output_image = cv2.addWeighted(cv2.cvtColor(img.astype('uint8'), cv2.COLOR_RGB2BGR), 0.5, cam, 1, 0)

cv2.imwrite('cam.png', output_image)

it would be great to provide some information on that

Hi, may I demand some clue about how to predict gradcam output not just a single image per time but also for multiple images at once?

Thanks

I have a trained 1d cnn model I would like use Grad CAM to see which part of inputs are most associated with output if you can give some brief explanation It would be great I am looking forward to hearing from you Hakan Karasu Sent from Mail<https://go.microsoft.com/fwlink/?LinkId=550986> for Windows From: Erol ***@***.***> Sent: Wednesday, September 1, 2021 7:27 AM To: ***@***.***> Cc: ***@***.***>; ***@***.***> Subject: Re: RaphaelMeudec/grad_cam.py @PhantomClick commented on this gist. Hi, may I demand some clue about how to predict gradcam output not just a single image per time but also for multiple images at once? Thanks — You are receiving this because you commented. Reply to this email directly, view it on GitHub<https://gist.github.com/e9a805fa82880876f8d89766f0690b54#gistcomment-3878552>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AUGXVYC2W32OPDA4KNQX4BDT7VJOPANCNFSM4LFEHWEA>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

Hi, may I demand some clue about how to predict gradcam output not just a single image per time but also for multiple images at once?

Thanks

The easiest way would be to put this code into a function, using the image path as argument (if you want to test different models, add them as argument as well). Then, simply create a 'for' loop for all the image paths you want to analyze.

Hi, may I demand some clue about how to predict gradcam output not just a single image per time but also for multiple images at once?
Thanks

The easiest way would be to put this code into a function, using the image path as argument (if you want to test different models, add them as argument as well). Then, simply create a 'for' loop for all the image paths you want to analyze.

Thank you for your response. Actually, I was wondering that what should I do or how should I approach this problem if I want to use my gpu to extract for example 20 images at once, otherwise in for loop solution I have to feed every image one by one.

Hi, may I demand some clue about how to predict gradcam output not just a single image per time but also for multiple images at once?
Thanks

The easiest way would be to put this code into a function, using the image path as argument (if you want to test different models, add them as argument as well). Then, simply create a 'for' loop for all the image paths you want to analyze.

Thank you for your response. Actually, I was wondering that what should I do or how should I approach this problem if I want to use my gpu to extract for example 20 images at once, otherwise in for loop solution I have to feed every image one by one.

Create a list containing the image paths for the 20 images you want to analyze, and then iterate on that list using the 'for' loop.

Hello and thank you very much for this code, it really saved me :)
I have a question regarding CAT_CLASS_INDEX = 281. What does it mean? How do you get that value? And what changes I may have to do if I want to implement this code to dogs pictures or for example medicine images?

The model is a pretrained VGG that has been trained on imagenet. Imagenet has 1000 classes (the last layer of the VGG is a dense one with 1000 outputs), and the index that has been used during training for cat is the 281-th. The list is available here.

For your case, you just need to know which index is used for the "dog" and then pass the according index.

I see! Thank you very much :)

How would you find the feature importance for in terms of the RGB colour channels? Here you get pixel importance, which is a combination of the RGB input. I'm very interest in the answer as it relates to a problem I have where I want to find the grad cam for a multivariate time series

@RaphaelMeudec Hi. what if the input size is 192,152,4 where 4 is number of 2D slices? how can we obtain for each of the image??