Skip to content

Instantly share code, notes, and snippets.

@gongzhitaao
Last active March 27, 2017 18:47
Show Gist options
  • Save gongzhitaao/e56ba03607eccafc616ca9677f8d9ba6 to your computer and use it in GitHub Desktop.
Save gongzhitaao/e56ba03607eccafc616ca9677f8d9ba6 to your computer and use it in GitHub Desktop.
A keras bug reproduce.

The script shows a wierd bug using keras with tensorflow.

When you use the tf.while_loop function

  1. if you include the Dropout layer or set the total number of epochs to greater than 10, then the program hangs and never returns.
  2. Otherwise, it works fine.

I've updated to the latest version of keras and tensorflow. This bug persists. It seems to be a bug with keras. Since I could not reproduce this with pure tensorflow code.

import os
# supress tensorflow logging other than errors
os.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'
mport numpy as np
import tensorflow as tf
import keras
from keras import backend as K
from keras.models import Sequential
from keras.layers import Dense, Dropout
print(keras.__version__)
print(tf.__version__)
sess = tf.InteractiveSession()
K.set_session(sess)
model = Sequential([
Dense(10, input_shape=[3,]),
Dropout(0.5), # comment out this line and it works
Dense(1)])
model.compile(optimizer='adam', loss='binary_crossentropy',
metrics=['accuracy'])
nb_epoch = tf.constant(20)
def _cond(xx, i):
return i < nb_epoch
def _body(xx, i):
y = model(xx)
dy_dx, = tf.gradients(y, xx)
xx = tf.stop_gradient(xx + tf.sign(dy_dx))
i += 1
return xx, i
x = tf.placeholder(tf.float32, (None, 3))
xx = tf.identity(x)
i = tf.Variable(0)
xx, i = tf.while_loop(_cond, _body, [xx, i])
init = tf.global_variables_initializer()
sess.run(init)
val = sess.run(xx, feed_dict={
x: np.random.random((1, 3)),
K.learning_phase(): 0
})
print(val)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment