Skip to content

Instantly share code, notes, and snippets.

View siemanko's full-sized avatar

Szymon Sidor siemanko

358 followers · 2 following
  • OpenAI
  • San Franscisco, CA
View GitHub Profile
@siemanko
siemanko / tf_lstm.py
Last active July 26, 2023 06:57
Simple implementation of LSTM in Tensorflow in 50 lines (+ 130 lines of data generation and comments)
"""Short and sweet LSTM implementation in Tensorflow.
Motivation:
When Tensorflow was released, adding RNNs was a bit of a hack - it required
building separate graphs for every number of timesteps and was a bit obscure
to use. Since then TF devs added things like `dynamic_rnn`, `scan` and `map_fn`.
Currently the APIs are decent, but all the tutorials that I am aware of are not
making the best use of the new APIs.
Advantages of this implementation:
@siemanko
siemanko / my_lstm_cell.py
Last active April 21, 2020 11:19
Simplified version of LSTM cell. Adds an option to learn zero_state
class MyLSTMCell(tf.nn.rnn_cell.RNNCell):
"""Simplified Version rnn_cell.BasicLSTMCell"""
def __init__(self, num_units):
super(MyLSTMCell, self).__init__()
self._num_units = num_units
def __call__(self, inputs, state, scope="LSTM"):
with tf.variable_scope(scope):
c, h = state
@siemanko
siemanko / initialization_trouble.py
Created March 11, 2017 04:55
import argparse
import numpy as np
import tensorflow as tf
from tensorflow.python.framework.errors import FailedPreconditionError
"""Code for data dependent initialization in Weight Normalization paper:
https://arxiv.org/abs/1602.07868
"""
@siemanko
siemanko / codenames.html
Last active July 4, 2017 13:50
Codenames
<html>
<head>
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap.min.css" integrity="sha384-BVYiiSIFeK1dGmJRAkycuHAHRg32OmUcww7on3RYdg4Va+PmSTsz/K68vbdEjh4u" crossorigin="anonymous">
<link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/3.3.7/css/bootstrap-theme.min.css" integrity="sha384-rHyoN1iRsVXV4nD0JutlnGaslCJuC7uwjduW9SVrLvRYooPp2bWYgmgJQIXwl/Sp" crossorigin="anonymous">
<style>
body {
padding-top: 70px;
}
.cards {
vertical-align: middle;