Skip to content

Instantly share code, notes, and snippets.

@Irio
Created July 7, 2019 08:27
Show Gist options
  • Save Irio/d00b9661023923be7c963395483dfd73 to your computer and use it in GitHub Desktop.
Save Irio/d00b9661023923be7c963395483dfd73 to your computer and use it in GitHub Desktop.
Elman network
Display the source blob
Display the rendered blob
Raw
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@Irio
Copy link
Author

Irio commented Jul 7, 2019

graph

@Irio
Copy link
Author

Irio commented Jul 9, 2019

After changes suggested by jdehesa:

# Input units

x = tf.placeholder(tf.float32, shape=(n_0, m), name="x")
y = tf.placeholder(tf.float32, shape=(n_2, m), name="y")
w_x = tf.Variable(parameters["w_x"], name="w_x")

# Context units

w_c = tf.Variable(parameters["w_c"], name="w_c")
a_c = tf.Variable(parameters["a_c"], trainable=False, name="a_c")

# First layer

a_0 = tf.concat([x, a_c], 0, name="a_0")

# Hidden units

w_1 = tf.concat([w_x, w_c], 1, name="w_1")
b_1 = tf.Variable(parameters["b_1"], name="b_1")
z_1 = tf.add(tf.matmul(w_1, a_0), b_1, name="z_1")
a_1 = tf.sigmoid(z_1)
with tf.control_dependencies([tf.assign(a_c, a_1)]):
    a_1 = tf.identity(a_1)

# Output units

w_2 = tf.Variable(parameters["w_2"], name="w_2")
b_2 = tf.Variable(parameters["b_2"], name="b_2")
z_2 = tf.add(tf.matmul(w_2, a_1), b_2, name="z_2")
a_2 = tf.sigmoid(z_2, name="a_2")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment