Skip to content

Instantly share code, notes, and snippets.

View dwf's full-sized avatar

David Warde-Farley dwf

View GitHub Profile
@dwf
dwf / gist:3790284
Created September 26, 2012 20:12
A key-aware default dictionary implementation.
class KeyAwareDefaultDict(dict):
"""
Like a standard library defaultdict, but pass the key
to the default factory.
"""
def __init__(self, default_factory=None):
self.default_factory = default_factory
def __getitem__(self, key):
if key not in self and self.default_factory is not None:
@dwf
dwf / kmeans.pyx
Created March 25, 2012 22:18
Parallelized k-means in Cython.
"""
Parallelized k-means module.
By David Warde-Farley, February 2012. Licensed under the 3-clause BSD.
"""
cimport cython
from cython.parallel import prange
import numpy as np
cimport numpy as np
from numpy.random import normal
@dwf
dwf / gist:828099
Created February 15, 2011 19:50
Pull out a submatrix from a COO SciPy sparse matrix.
import scipy as S
def coo_submatrix_pull(matr, rows, cols):
"""
Pulls out an arbitrary i.e. non-contiguous submatrix out of
a sparse.coo_matrix.
"""
if type(matr) != S.sparse.coo_matrix:
raise TypeError('Matrix must be sparse COOrdinate format')
@dwf
dwf / coventry_regex.py
Created August 28, 2018 18:00
Autofill CoventryBS's silly 2FA with Bitwarden regex custom fields.
"""
Take in a CoventryBS grid card as a text file, output regex custom fields for
Bitwarden to autofill your grid card challenge for you.
"""
from __future__ import print_function
import fileinput
import itertools
import sys
COLS = 'ABCDEFGHIJ'
@dwf
dwf / hinton.py
Created February 1, 2010 20:51
A function for drawing Hinton diagrams with matplotlib.
#!/usr/bin/env python
"""
Draws Hinton diagrams using matplotlib ( http://matplotlib.sf.net/ ).
Hinton diagrams are a handy way of visualizing weight matrices, using
colour to denote sign and area to denote magnitude.
By David Warde-Farley -- user AT cs dot toronto dot edu (user = dwf)
with thanks to Geoffrey Hinton for providing the MATLAB code off of
which this is modeled.
@dwf
dwf / bcefl.py
Created April 10, 2017 16:00
Stable binary cross-entropy, operating on logit predictions instead of relying on Theano to correctly optimize the sigmoid output.
from theano import tensor
def binary_crossentropy_from_logits(logits, targets):
"""Binary cross-entropy computed from model logits.
Parameters
----------
predictions : TensorVariable
The unnormalized log probabilities of a probabilistic binary
classifier.
targets : TensorVariable
@dwf
dwf / rocarea.py
Created August 30, 2010 19:01
ROC curve plotting code.
"""
Simple implementation of ROC curve plotting with NumPy and matplotlib.
No bells and whistles, no fancy data structures, just one function and
a (hopefully) very gentle learning curve.
"""
__author__ = "David Warde-Farley <dwf AT cs.toronto.edu>"
__copyright__ = "(c) 2010 David Warde-Farley"
__license__ = "3-clause BSD license"
@dwf
dwf / reader.pyx
Last active February 10, 2017 08:45
A proof-of-concept for a multi-threaded pre-fetching loader for .NPY files.
"""
A proof-of-concept multi-threaded chunked NPY format reader.
"""
__author__ = "David Warde-Farley"
__credits__ = ["David Warde-Farley"]
__license__ = "3-clause BSD"
__email__ = 'd dot warde dot farley at gmail DOT com"
import threading
from numpy.lib.format import read_magic, read_array_header_1_0
@dwf
dwf / simple_mnist_gan.py
Created January 20, 2017 14:27
Single-script Blocks reproduction of the Goodfellow et al (2014) MNIST GAN.
from collections import OrderedDict
from blocks.algorithms import GradientDescent, Momentum
from blocks.bricks import (
MLP, Rectifier, Logistic, Linear, LinearMaxout, Sequence, Random
)
from blocks.extensions import Printing, Timing
from blocks.extensions.training import SharedVariableModifier
from blocks.extensions.monitoring import (DataStreamMonitoring,
TrainingDataMonitoring)
@dwf
dwf / unhexdump.py
Created November 13, 2013 22:07
Undo a textual hex dump (from the Firefox cache).
"""
Reverses a hex dump (of the format shown in the Firefox cache).
The format is as follows:
<OFFSET> <BYTE> [<BYTE> [<BYTE> [...]]] [render]
Where
- <OFFSET> is 8 hex digits, possibly followed by a colon
- each <BYTE> is (maximum 16 per line)