Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Tensorflow: visualize convolutional filters (conv1) in Cifar10 model
from math import sqrt
def put_kernels_on_grid (kernel, pad = 1):
'''Visualize conv. filters as an image (mostly for the 1st layer).
Arranges filters into a grid, with some paddings between adjacent filters.
Args:
kernel: tensor of shape [Y, X, NumChannels, NumKernels]
pad: number of black pixels around each filter (between them)
Return:
Tensor of shape [1, (Y+2*pad)*grid_Y, (X+2*pad)*grid_X, NumChannels].
'''
# get shape of the grid. NumKernels == grid_Y * grid_X
def factorization(n):
for i in range(int(sqrt(float(n))), 0, -1):
if n % i == 0:
if i == 1: print('Who would enter a prime number of filters')
return (i, int(n / i))
(grid_Y, grid_X) = factorization (kernel.get_shape()[3].value)
print ('grid: %d = (%d, %d)' % (kernel.get_shape()[3].value, grid_Y, grid_X))
x_min = tf.reduce_min(kernel)
x_max = tf.reduce_max(kernel)
kernel = (kernel - x_min) / (x_max - x_min)
# pad X and Y
x = tf.pad(kernel, tf.constant( [[pad,pad],[pad, pad],[0,0],[0,0]] ), mode = 'CONSTANT')
# X and Y dimensions, w.r.t. padding
Y = kernel.get_shape()[0] + 2 * pad
X = kernel.get_shape()[1] + 2 * pad
channels = kernel.get_shape()[2]
# put NumKernels to the 1st dimension
x = tf.transpose(x, (3, 0, 1, 2))
# organize grid on Y axis
x = tf.reshape(x, tf.stack([grid_X, Y * grid_Y, X, channels]))
# switch X and Y axes
x = tf.transpose(x, (0, 2, 1, 3))
# organize grid on X axis
x = tf.reshape(x, tf.stack([1, X * grid_X, Y * grid_Y, channels]))
# back to normal order (not combining with the next step for clarity)
x = tf.transpose(x, (2, 1, 3, 0))
# to tf.image_summary order [batch_size, height, width, channels],
# where in this case batch_size == 1
x = tf.transpose(x, (3, 0, 1, 2))
# scaling to [0, 255] is not necessary for tensorboard
return x
#
# ... and somewhere inside "def train():" after calling "inference()"
#
# Visualize conv1 kernels
with tf.variable_scope('conv1'):
tf.get_variable_scope().reuse_variables()
weights = tf.get_variable('weights')
grid = put_kernels_on_grid (weights)
tf.image.summary('conv1/kernels', grid, max_outputs=1)
@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Mar 8, 2016

That's how it looks

@twimnox

This comment has been minimized.

Copy link

@twimnox twimnox commented Mar 9, 2016

I used your code in my cifar10 model but i'm having some issues.

  1. I placed the #Visualize conv1 featurespiece of code just before
 # Build an initialization operation to run below.
    init = tf.initialize_all_variables()

and I got the following error:

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 71, in get_variable " Did you mean to set reuse=True in VarScope?" % name) ValueError: Over-sharing: Variable conv1/weights already exists, disallowed. Did you mean to set reuse=True in VarScope?

  1. So I did this modification:
`    # Visualize conv1 features
    with tf.variable_scope('conv1') as scope_conv:
        tf.get_variable_scope().reuse_variables()`

and now I'm having this error:
File "/home/prtricardo/tese_ws/open_cv/acacia_model/cifar10_train.py", line 86, in put_kernels_on_grid return tf.image.convert_image_dtype(x8, dtype=tf.uint8) AttributeError: 'module' object has no attribute 'convert_image_dtype'

Do you have any clue of what this might be?

@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Mar 19, 2016

Hey, @twimnox,

  1. you are absolutely right. I used that line before for my needs, so I just forgot to copypaste it.
  2. I don't get that error, but in any case, apparently tensorflow can display float-type images in range [0, 1] too. So we can just get rid of tf.image.convert_image_dtype(x8, dtype=tf.uint8) altogether.
    I updated the code - can you try it?
@lotka

This comment has been minimized.

Copy link

@lotka lotka commented May 29, 2016

Took me awhile to see that I had to change those 3's with 1's because I have greyscale images. But otherwise it works well thank you. Do you know how to generalise it to intermediate convolutional filters?

@ifishlin

This comment has been minimized.

Copy link

@ifishlin ifishlin commented Sep 6, 2016

Hey @kukuruza

I put the put_kernels_on_grid() before def train() in cifar10_train.py
and placed the short script code just before

#Build an initialization operation to run below.
init = tf.initialize_all_variables()

however, I got nothing on my tensorboad. (no error msg too)
I tried the code you shared on stackoverflow, that one work well.
(http://stackoverflow.com/questions/35759220/how-to-visualize-learned-filters-on-tensorflow)
Do you have any suggestion? thanks

@kitovyj

This comment has been minimized.

Copy link

@kitovyj kitovyj commented Sep 21, 2016

I modified your code a liitle to make it compatible with arbitrary channel number and to get rid of the variable background color(normalizatoin is done berofe the padding):

def put_kernels_on_grid (kernel, grid_Y, grid_X, pad = 1):

    '''Visualize conv. features as an image (mostly for the 1st layer).
    Place kernel into a grid, with some paddings between adjacent filters.

    Args:
      kernel:            tensor of shape [Y, X, NumChannels, NumKernels]
      (grid_Y, grid_X):  shape of the grid. Require: NumKernels == grid_Y * grid_X
                           User is responsible of how to break into two multiples.
      pad:               number of black pixels around each filter (between them)

    Return:
      Tensor of shape [(Y+2*pad)*grid_Y, (X+2*pad)*grid_X, NumChannels, 1].
    '''

    x_min = tf.reduce_min(kernel)
    x_max = tf.reduce_max(kernel)

    kernel1 = (kernel - x_min) / (x_max - x_min)

    # pad X and Y
    x1 = tf.pad(kernel1, tf.constant( [[pad,pad],[pad, pad],[0,0],[0,0]] ), mode = 'CONSTANT')

    # X and Y dimensions, w.r.t. padding
    Y = kernel1.get_shape()[0] + 2 * pad
    X = kernel1.get_shape()[1] + 2 * pad

    channels = kernel1.get_shape()[2]

    # put NumKernels to the 1st dimension
    x2 = tf.transpose(x1, (3, 0, 1, 2))
    # organize grid on Y axis
    x3 = tf.reshape(x2, tf.pack([grid_X, Y * grid_Y, X, channels])) #3

    # switch X and Y axes
    x4 = tf.transpose(x3, (0, 2, 1, 3))
    # organize grid on X axis
    x5 = tf.reshape(x4, tf.pack([1, X * grid_X, Y * grid_Y, channels])) #3

    # back to normal order (not combining with the next step for clarity)
    x6 = tf.transpose(x5, (2, 1, 3, 0))

    # to tf.image_summary order [batch_size, height, width, channels],
    #   where in this case batch_size == 1
    x7 = tf.transpose(x6, (3, 0, 1, 2))

    # scale to [0, 255] and convert to uint8
    return tf.image.convert_image_dtype(x7, dtype = tf.uint8) 
@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Sep 22, 2016

Thanks for your code, @kitovyj! Makes sense that the bckground should be black. I'll update the gist when I have a chance to test it

@kootenpv

This comment has been minimized.

Copy link

@kootenpv kootenpv commented Oct 11, 2016

Guys, awesome work! I can confirm @kitovyj code works. Actually, the original version on line 1 contains a typo (tuple in the arguments).

Is there a way to show the original image right next to an image after the convolution layer?

@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Dec 29, 2016

Updated:

  • grayscale or color filters (thanks to @kitovyj)
  • optimal grid shape is computed automatically
@JiteshPshah

This comment has been minimized.

Copy link

@JiteshPshah JiteshPshah commented Apr 14, 2017

How to show it in a tensorboard ? what is the command ?

@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Apr 15, 2017

@JiteshPshah. Start your tensorboard as usual (see TF tutorials), then go to the image tab. It should be there.

@yaxingwang

This comment has been minimized.

Copy link

@yaxingwang yaxingwang commented Apr 20, 2017

@kukuruza the title is visualize convolutional features (conv1), but the code is about kernel which is not feature. forgive me about the detail information.

@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Apr 28, 2017

@yaxingwang True.

@llewynS

This comment has been minimized.

Copy link

@llewynS llewynS commented May 2, 2017

This doesn't enforce that the last number be in the range 1-3 which means that it can cause errors.

@Mithun-Aggarwal

This comment has been minimized.

Copy link

@Mithun-Aggarwal Mithun-Aggarwal commented Nov 7, 2017

This was extremely useful guys.
image

But is there any way I can view other conv layers in my Gen & discriminator

as when I try I get the error:
InvalidArgumentError (see above for traceback): Tensor must be 4-D with last dim 1, 3, or 4, not [1,112,224,256]
[[Node: generator/conv_tranpose1/Conv_filter_2 = ImageSummary[T=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=1, _device="/job:localhost/replica:0/task:0/cpu:0"](generator/conv_tranpose1/Conv_filter_2/tag, generator/conv_tranpose1/transpose_3)]]

hope i could help impove this snippet.

@2016gary

This comment has been minimized.

Copy link

@2016gary 2016gary commented Nov 13, 2017

@Mithun-Aggarwal I also encountered the same problem, do you have a solution?

@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented Jan 27, 2018

@Mithun-Aggarwal, @2016gary. Sorry, I didn't check the comments in a while.
It does not make much sense to visualize conv layer 2+, except maybe for checking for dead filters.
Besides, humans can visualize 3 channels as RGB, but will not be able to interpret 64 channels.

Now, if you're really into it, here's what you can do. Let's say you have 32 input channels on the 2nd layer out 64 output channels. Instead of having one grid with 64 color squares, make 64 grids with 32 grayscale channels. Each grid == a filter. Again, it would be be hard to interpret, but easy to spot dead filters.
I may implement that some time in the future.

@headdab

This comment has been minimized.

Copy link

@headdab headdab commented Apr 30, 2018

The code by @kitovyj should use tf.stack, instead of tf.pack.

@kukuruza

This comment has been minimized.

Copy link
Owner Author

@kukuruza kukuruza commented May 4, 2018

@headdab, I already implemented changes suggested by @kitovyj into this gist, and changed pack to stack according to API of the recent TF versions.

@IzPine

This comment has been minimized.

Copy link

@IzPine IzPine commented Mar 3, 2020

 raise ValueError("Variable %s does not exist, or was not created with "
                 "tf.get_variable(). Did you mean to set "
                  "reuse=tf.AUTO_REUSE in VarScope?" % name)

Create the tensor to initialize the variable with default value.

ValueError: Variable conv1/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

I got this error. What should I do?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment