Skip to content

Instantly share code, notes, and snippets.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@timvieira
timvieira / Notes-on-scipy.special.digamma.ipynb
Last active May 29, 2023 13:46
Don't use scipy.special.digamma, if you care about speed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@timvieira
timvieira / lagrangeprop.py
Last active May 1, 2022 06:54
Automatic differentiation as the method of Lagrange multipliers. Code accompanies this blog post: http://timvieira.github.io/blog/post/2017/08/18/backprop-is-not-just-the-chain-rule/
# -*- coding: utf-8 -*-
"""
Backprop as the method of Lagrange multiplers (and even the implicit function
theorem).
"""
from __future__ import division
import numpy as np
from arsenal.alphabet import Alphabet
from arsenal.math.checkgrad import finite_difference
@timvieira
timvieira / simple-backprop.py
Last active May 14, 2022 04:32
Simple example of manually performing "automatic" differentiation.
"""
Simple example of manually performing "automatic" differentiation
"""
import numpy as np
from numpy import exp, sin, cos
def f(x, with_grad=False):
# Need to cache intermediates from forward pass (might not use all of them).
a = exp(x)
# Efficient passive aggressive updates for multi-class classification
#
# Original article:
# "Column squishing for multiclass updates"
# https://nlpers.blogspot.com/2017/08/column-squishing-for-multiclass-updates.html
from __future__ import division
import numpy as np
import scipy.optimize
"""
Memory-efficient backpropagation in an RNN.
Accompanies blog post:
http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/
"""
import numpy as np
from arsenal.math.checkgrad import fdcheck
@timvieira
timvieira / memory-efficient-backprop.py
Created August 8, 2017 17:41
Memory efficient backpropagation thru time in a recurrent neural network. Accompanies blog post: http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/
"""
Memory-efficient backpropagation in an RNN.
Accompanies blog post:
http://timvieira.github.io/blog/post/2016/10/01/reversing-a-sequence-with-sublinear-space/
"""
import numpy as np
from arsenal.math.checkgrad import fdcheck
@timvieira
timvieira / jiawei.py
Created February 18, 2017 20:18
Cartoon version of Jiawei's optimization problem.
"""
Cartoon version of Jiawei's optimization problem.
Created [2017-02-17 Fri]
"""
import numpy as np
from scipy.optimize import fmin_bfgs
import autograd
@timvieira
timvieira / counterfactual-demo.ipynb
Last active May 12, 2022 05:25
Counterfactual reasoning demo. Accompanies blog post "Counterfactual reasoning and learning from logged data" http://timvieira.github.io/blog/post/2016/12/19/counterfactual-reasoning-and-learning-from-logged-data/
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@timvieira
timvieira / heap-sample.py
Created November 26, 2016 19:21
Fast sampling from an evolving distribution
import numpy as np
from numpy.random import uniform
def update(S, k, v):
"Update value position `k` in time O(log n)."
d = S.shape[0]
i = d//2 + k
S[i] = v
while i > 0: