Navigation Menu

Skip to content

Instantly share code, notes, and snippets.

View dwf's full-sized avatar

David Warde-Farley dwf

View GitHub Profile
@dwf
dwf / docscrape.py
Created February 25, 2014 19:53
Code extracted from the numpydoc package.
"""Extract reference documentation from the NumPy source tree.
"""
import inspect
import textwrap
import re
import sys
import types
@dwf
dwf / gist:10165796
Created April 8, 2014 18:15
Dropout yaml file.
# These hyperparameters, identified by David Warde-Farley via random search,
# obtain 1.08% on MNIST, nearly matching the 1.05% in Nitish Srivastava's
# Master's thesis.
#
# An important difference is that this uses only the first 50,000 examples
# and does early stopping on a validation set of the last 10,000 training
# points, whereas Nitish Srivastava's results trained on the entire training
# set for a given number of epochs. It is quite possible that re-training
# with these hyperparameters on the entire 60,000 using some alternate stopping
# criterion (matching the best training set likelihood of this job, for instance,
from functools import wraps
import sys
# Goddamnit Python 2/3 differences.
str_type = str if sys.version_info[0] >= '3' else basestring
def filename_or_file_like(mode):
"""Decorator that checks if the first argument to a function is
@dwf
dwf / gist:fd13ca098b1cb45e9011
Created February 24, 2015 22:03
Multithreaded buffering NPY reader.
try:
from queue import Queue, Empty
except ImportError:
from Queue import Queue, Empty
import threading
import numpy
from numpy.lib.format import read_magic, read_array_header_1_0
class BufferedChunkedNPY(object):
@dwf
dwf / gist:83f79c0533a26f56d09a
Last active August 29, 2015 14:19
Proof of concept using a logging handler and LogRecord "extras" attribute to manage a progress bar.
# Copyright (c) 2015 David Warde-Farley.
#
# Permission is granted to use this code under the MIT license:
# http://opensource.org/licenses/mit-license.php
import logging
import progressbar
import time
@dwf
dwf / mobile-shell-mousefix.rb
Created May 25, 2015 21:17
Homebrew formula for a version of mosh with xterm mouse reporting fixed (i.e. works more than once per tmux session)
class MobileShellMousefix < Formula
homepage "http://mosh.mit.edu/"
revision 2
head do
url "https://github.com/lpkruger/mosh.git", :revision => "d5f75ec54a"
depends_on "autoconf" => :build
depends_on "automake" => :build
end
"""Very simple PUSH-PULL reusable producer-consumer with ZeroMQ."""
# By David Warde-Farley. Released under the 3-clause BSD license.
import time
from multiprocessing import Process
import zmq
def _producer_wrapper(f, port, addr='tcp://127.0.0.1'):
@dwf
dwf / uses_random.py
Created January 27, 2010 09:35
Object-specific NumPy random number generator states
"""
A demonstration of how to deal with object-specific random number
generator states. One application is if you want two objects to
work with the same pseudorandom sequence but don't particularly
want to generate them in advance, or to replicate results after
serializing and de-serializing an object.
By David Warde-Farley, dwf at cs.toronto.edu, January 2010.
Released under BSD license.
"""
@dwf
dwf / pfball.py
Created January 27, 2010 22:25
My Python rewrite of Brendan Frey's product form sampling code.
import numpy as np
def sample_pf_ball(p,b):
"""
Samples N binary variables using the product form distribution given in
the vector b (of length N), where b[n] is the probability that s[n]=1.
However, only K variables may on at a time, where K is the length of
vector b minus 1, and b[k] is a distribution that biases the number of
variables set to 1.
%
@dwf
dwf / fourregions.py
Created February 1, 2010 20:57
Singhal & Wu's four regions benchmark for nonlinear classification algorithms, in Python/NumPy.
#!/usr/bin/env python
"""
NumPy implementation of the classic 'four regions' benchmark.
By David Warde-Farley -- user AT cs dot toronto dot edu (user = dwf)
Redistributable under the terms of the 3-clause BSD license
(see http://www.opensource.org/licenses/bsd-license.php for details)
"""
import numpy as np