Szymon Sidor siemanko
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <html> | |
| <head> | |
| <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous"> | |
| <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous"> | |
| <style> | |
| body { | |
| padding-top: 70px; | |
| } | |
| .cards { | |
| vertical-align: middle; |
siemanko
/ initialization_trouble.py
Created
March 11, 2017 04:55
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import argparse | |
| import numpy as np | |
| import tensorflow as tf | |
| from tensorflow.python.framework.errors import FailedPreconditionError | |
| """Code for data dependent initialization in Weight Normalization paper: | |
| https://arxiv.org/abs/1602.07868 | |
| """ |
siemanko
/ my_lstm_cell.py
Last active
April 21, 2020 11:19
Simplified version of LSTM cell. Adds an option to learn zero_state
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| class MyLSTMCell(tf.nn.rnn_cell.RNNCell): | |
| """Simplified Version rnn_cell.BasicLSTMCell""" | |
| def __init__(self, num_units): | |
| super(MyLSTMCell, self).__init__() | |
| self._num_units = num_units | |
| def __call__(self, inputs, state, scope="LSTM"): | |
| with tf.variable_scope(scope): | |
| c, h = state |
siemanko
/ tf_lstm.py
Last active
July 26, 2023 06:57
Simple implementation of LSTM in Tensorflow in 50 lines (+ 130 lines of data generation and comments)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| """Short and sweet LSTM implementation in Tensorflow. | |
| Motivation: | |
| When Tensorflow was released, adding RNNs was a bit of a hack - it required | |
| building separate graphs for every number of timesteps and was a bit obscure | |
| to use. Since then TF devs added things like `dynamic_rnn`, `scan` and `map_fn`. | |
| Currently the APIs are decent, but all the tutorials that I am aware of are not | |
| making the best use of the new APIs. | |
| Advantages of this implementation: |