Instantly share code, notes, and snippets.

# Tim Vieira timvieira

Last active June 12, 2019 02:25
Notebook accompanying blog post http://timvieira.github.io/blog/post/2019/06/11/faster-reservoir-sampling-by-waiting/
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Last active May 29, 2023 13:46
Don't use scipy.special.digamma, if you care about speed.
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Last active May 1, 2022 06:54
Automatic differentiation as the method of Lagrange multipliers. Code accompanies this blog post: http://timvieira.github.io/blog/post/2017/08/18/backprop-is-not-just-the-chain-rule/
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 # -*- coding: utf-8 -*- """ Backprop as the method of Lagrange multiplers (and even the implicit function theorem). """ from __future__ import division import numpy as np from arsenal.alphabet import Alphabet from arsenal.math.checkgrad import finite_difference
Last active May 14, 2022 04:32
Simple example of manually performing "automatic" differentiation.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 """ Simple example of manually performing "automatic" differentiation """ import numpy as np from numpy import exp, sin, cos def f(x, with_grad=False): # Need to cache intermediates from forward pass (might not use all of them). a = exp(x)
Last active August 22, 2017 02:40
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 # Efficient passive aggressive updates for multi-class classification # # Original article: # "Column squishing for multiclass updates" # https://nlpers.blogspot.com/2017/08/column-squishing-for-multiclass-updates.html from __future__ import division import numpy as np import scipy.optimize
Created August 8, 2017 17:43
Memory-efficient backpropagation in an RNN. Accompanies blog post: http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 """ Memory-efficient backpropagation in an RNN. Accompanies blog post: http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/ """ import numpy as np from arsenal.math.checkgrad import fdcheck
Created August 8, 2017 17:41
Memory efficient backpropagation thru time in a recurrent neural network. Accompanies blog post: http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 """ Memory-efficient backpropagation in an RNN. Accompanies blog post: http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/ """ import numpy as np from arsenal.math.checkgrad import fdcheck
Created February 18, 2017 20:18
Cartoon version of Jiawei's optimization problem.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
 """ Cartoon version of Jiawei's optimization problem. Created [2017-02-17 Fri] """ import numpy as np from scipy.optimize import fmin_bfgs import autograd
Last active May 12, 2022 05:25
Counterfactual reasoning demo. Accompanies blog post "Counterfactual reasoning and learning from logged data" http://timvieira.github.io/blog/post/2016/12/19/counterfactual-reasoning-and-learning-from-logged-data/