Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Tensorflow: visualize convolutional filters (conv1) in Cifar10 model
from math import sqrt
def put_kernels_on_grid (kernel, pad = 1):
'''Visualize conv. filters as an image (mostly for the 1st layer).
Arranges filters into a grid, with some paddings between adjacent filters.
Args:
kernel: tensor of shape [Y, X, NumChannels, NumKernels]
pad: number of black pixels around each filter (between them)
Return:
Tensor of shape [1, (Y+2*pad)*grid_Y, (X+2*pad)*grid_X, NumChannels].
'''
# get shape of the grid. NumKernels == grid_Y * grid_X
def factorization(n):
for i in range(int(sqrt(float(n))), 0, -1):
if n % i == 0:
if i == 1: print('Who would enter a prime number of filters')
return (i, int(n / i))
(grid_Y, grid_X) = factorization (kernel.get_shape()[3].value)
print ('grid: %d = (%d, %d)' % (kernel.get_shape()[3].value, grid_Y, grid_X))
x_min = tf.reduce_min(kernel)
x_max = tf.reduce_max(kernel)
kernel = (kernel - x_min) / (x_max - x_min)
# pad X and Y
x = tf.pad(kernel, tf.constant( [[pad,pad],[pad, pad],[0,0],[0,0]] ), mode = 'CONSTANT')
# X and Y dimensions, w.r.t. padding
Y = kernel.get_shape()[0] + 2 * pad
X = kernel.get_shape()[1] + 2 * pad
channels = kernel.get_shape()[2]
# put NumKernels to the 1st dimension
x = tf.transpose(x, (3, 0, 1, 2))
# organize grid on Y axis
x = tf.reshape(x, tf.stack([grid_X, Y * grid_Y, X, channels]))
# switch X and Y axes
x = tf.transpose(x, (0, 2, 1, 3))
# organize grid on X axis
x = tf.reshape(x, tf.stack([1, X * grid_X, Y * grid_Y, channels]))
# back to normal order (not combining with the next step for clarity)
x = tf.transpose(x, (2, 1, 3, 0))
# to tf.image_summary order [batch_size, height, width, channels],
# where in this case batch_size == 1
x = tf.transpose(x, (3, 0, 1, 2))
# scaling to [0, 255] is not necessary for tensorboard
return x
#
# ... and somewhere inside "def train():" after calling "inference()"
#
# Visualize conv1 kernels
with tf.variable_scope('conv1'):
tf.get_variable_scope().reuse_variables()
weights = tf.get_variable('weights')
grid = put_kernels_on_grid (weights)
tf.image.summary('conv1/kernels', grid, max_outputs=1)
@kukuruza
Copy link
Author

kukuruza commented Mar 8, 2016

That's how it looks

@twimnox
Copy link

twimnox commented Mar 9, 2016

I used your code in my cifar10 model but i'm having some issues.

  1. I placed the #Visualize conv1 featurespiece of code just before
 # Build an initialization operation to run below.
    init = tf.initialize_all_variables()

and I got the following error:

File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/variable_scope.py", line 71, in get_variable " Did you mean to set reuse=True in VarScope?" % name) ValueError: Over-sharing: Variable conv1/weights already exists, disallowed. Did you mean to set reuse=True in VarScope?

  1. So I did this modification:
`    # Visualize conv1 features
    with tf.variable_scope('conv1') as scope_conv:
        tf.get_variable_scope().reuse_variables()`

and now I'm having this error:
File "/home/prtricardo/tese_ws/open_cv/acacia_model/cifar10_train.py", line 86, in put_kernels_on_grid return tf.image.convert_image_dtype(x8, dtype=tf.uint8) AttributeError: 'module' object has no attribute 'convert_image_dtype'

Do you have any clue of what this might be?

@kukuruza
Copy link
Author

Hey, @twimnox,

  1. you are absolutely right. I used that line before for my needs, so I just forgot to copypaste it.
  2. I don't get that error, but in any case, apparently tensorflow can display float-type images in range [0, 1] too. So we can just get rid of tf.image.convert_image_dtype(x8, dtype=tf.uint8) altogether.
    I updated the code - can you try it?

@lotka
Copy link

lotka commented May 29, 2016

Took me awhile to see that I had to change those 3's with 1's because I have greyscale images. But otherwise it works well thank you. Do you know how to generalise it to intermediate convolutional filters?

@ifishlin
Copy link

ifishlin commented Sep 6, 2016

Hey @kukuruza

I put the put_kernels_on_grid() before def train() in cifar10_train.py
and placed the short script code just before

#Build an initialization operation to run below.
init = tf.initialize_all_variables()

however, I got nothing on my tensorboad. (no error msg too)
I tried the code you shared on stackoverflow, that one work well.
(http://stackoverflow.com/questions/35759220/how-to-visualize-learned-filters-on-tensorflow)
Do you have any suggestion? thanks

@kitovyj
Copy link

kitovyj commented Sep 21, 2016

I modified your code a liitle to make it compatible with arbitrary channel number and to get rid of the variable background color(normalizatoin is done berofe the padding):

def put_kernels_on_grid (kernel, grid_Y, grid_X, pad = 1):

    '''Visualize conv. features as an image (mostly for the 1st layer).
    Place kernel into a grid, with some paddings between adjacent filters.

    Args:
      kernel:            tensor of shape [Y, X, NumChannels, NumKernels]
      (grid_Y, grid_X):  shape of the grid. Require: NumKernels == grid_Y * grid_X
                           User is responsible of how to break into two multiples.
      pad:               number of black pixels around each filter (between them)

    Return:
      Tensor of shape [(Y+2*pad)*grid_Y, (X+2*pad)*grid_X, NumChannels, 1].
    '''

    x_min = tf.reduce_min(kernel)
    x_max = tf.reduce_max(kernel)

    kernel1 = (kernel - x_min) / (x_max - x_min)

    # pad X and Y
    x1 = tf.pad(kernel1, tf.constant( [[pad,pad],[pad, pad],[0,0],[0,0]] ), mode = 'CONSTANT')

    # X and Y dimensions, w.r.t. padding
    Y = kernel1.get_shape()[0] + 2 * pad
    X = kernel1.get_shape()[1] + 2 * pad

    channels = kernel1.get_shape()[2]

    # put NumKernels to the 1st dimension
    x2 = tf.transpose(x1, (3, 0, 1, 2))
    # organize grid on Y axis
    x3 = tf.reshape(x2, tf.pack([grid_X, Y * grid_Y, X, channels])) #3

    # switch X and Y axes
    x4 = tf.transpose(x3, (0, 2, 1, 3))
    # organize grid on X axis
    x5 = tf.reshape(x4, tf.pack([1, X * grid_X, Y * grid_Y, channels])) #3

    # back to normal order (not combining with the next step for clarity)
    x6 = tf.transpose(x5, (2, 1, 3, 0))

    # to tf.image_summary order [batch_size, height, width, channels],
    #   where in this case batch_size == 1
    x7 = tf.transpose(x6, (3, 0, 1, 2))

    # scale to [0, 255] and convert to uint8
    return tf.image.convert_image_dtype(x7, dtype = tf.uint8) 

@kukuruza
Copy link
Author

Thanks for your code, @kitovyj! Makes sense that the bckground should be black. I'll update the gist when I have a chance to test it

@kootenpv
Copy link

Guys, awesome work! I can confirm @kitovyj code works. Actually, the original version on line 1 contains a typo (tuple in the arguments).

Is there a way to show the original image right next to an image after the convolution layer?

@kukuruza
Copy link
Author

Updated:

  • grayscale or color filters (thanks to @kitovyj)
  • optimal grid shape is computed automatically

@JiteshPshah
Copy link

How to show it in a tensorboard ? what is the command ?

@kukuruza
Copy link
Author

@JiteshPshah. Start your tensorboard as usual (see TF tutorials), then go to the image tab. It should be there.

@yaxingwang
Copy link

@kukuruza the title is visualize convolutional features (conv1), but the code is about kernel which is not feature. forgive me about the detail information.

@kukuruza
Copy link
Author

@yaxingwang True.

@llewynS
Copy link

llewynS commented May 2, 2017

This doesn't enforce that the last number be in the range 1-3 which means that it can cause errors.

@Mithun-Aggarwal
Copy link

This was extremely useful guys.
image

But is there any way I can view other conv layers in my Gen & discriminator

as when I try I get the error:
InvalidArgumentError (see above for traceback): Tensor must be 4-D with last dim 1, 3, or 4, not [1,112,224,256]
[[Node: generator/conv_tranpose1/Conv_filter_2 = ImageSummary[T=DT_FLOAT, bad_color=Tensor<type: uint8 shape: [4] values: 255 0 0...>, max_images=1, _device="/job:localhost/replica:0/task:0/cpu:0"](generator/conv_tranpose1/Conv_filter_2/tag, generator/conv_tranpose1/transpose_3)]]

hope i could help impove this snippet.

@2016gary
Copy link

@Mithun-Aggarwal I also encountered the same problem, do you have a solution?

@kukuruza
Copy link
Author

@Mithun-Aggarwal, @2016gary. Sorry, I didn't check the comments in a while.
It does not make much sense to visualize conv layer 2+, except maybe for checking for dead filters.
Besides, humans can visualize 3 channels as RGB, but will not be able to interpret 64 channels.

Now, if you're really into it, here's what you can do. Let's say you have 32 input channels on the 2nd layer out 64 output channels. Instead of having one grid with 64 color squares, make 64 grids with 32 grayscale channels. Each grid == a filter. Again, it would be be hard to interpret, but easy to spot dead filters.
I may implement that some time in the future.

@headdab
Copy link

headdab commented Apr 30, 2018

The code by @kitovyj should use tf.stack, instead of tf.pack.

@kukuruza
Copy link
Author

kukuruza commented May 4, 2018

@headdab, I already implemented changes suggested by @kitovyj into this gist, and changed pack to stack according to API of the recent TF versions.

@IzzCode
Copy link

IzzCode commented Mar 3, 2020

 raise ValueError("Variable %s does not exist, or was not created with "
                 "tf.get_variable(). Did you mean to set "
                  "reuse=tf.AUTO_REUSE in VarScope?" % name)

Create the tensor to initialize the variable with default value.

ValueError: Variable conv1/weights does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=tf.AUTO_REUSE in VarScope?

I got this error. What should I do?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment