Skip to content

Instantly share code, notes, and snippets.

$ python hello.py
Topic apps daily avg: 620.02 MB over 31 days
Topic buzz_app daily avg: 538.45 MB over 31 days
Topic client_side_game_errors daily avg: 0.0 Byte over 0 days
Topic cms_events daily avg: 1.15 MB over 31 days
Topic conversion_events daily avg: 28.39 GB over 31 days
Topic cute_or_not_app daily avg: 2.16 KB over 31 days
Topic distributed_object_statistics daily avg: 1.83 KB over 3 days
Topic distributed_object_statistics_raw daily avg: 403.27 MB over 31 days
Topic distributed_objects_raw daily avg: 13.68 MB over 31 days
#!/usr/bin/env bash
# When eval'd, the output from script will export the AWS_ACCESS_KEY_ID and
# AWS_SECRET_ACCESS_KEY credentials from the a specific profile in
# ~/.aws/credentials.
#
# Usage:
#
# export-aws-credentials [PROFILE]
#
@mccutchen
mccutchen / cached.py
Last active November 25, 2015 17:11
Simple in-memory cache decorator with TTL
def cached(ttl, cache={}):
"""
A decorator for nullary functions that caches their results in memory for a
given number of seconds.
"""
def cached_decorator(f):
key = f.__name__
def decorated(*args, **kwargs):
if key in cache:
result, ts = cache[key]
@mccutchen
mccutchen / settings.py
Last active February 17, 2016 17:00
Skeleton settings.py used by data-infra squad
import json
import os
import socket
import tornado.options
tornado.options.define('environment', default='dev', help='environment')
app_name = os.path.basename(os.path.dirname(__file__))
@mccutchen
mccutchen / .gitconfig
Created June 4, 2015 15:59
My current ~/.gitconfig
[user]
name = Will McCutchen
email = will@mccutch.org
[alias]
br = branch
ci = commit
co = checkout
fb = "!git fetch origin && git rebase origin/master $@"
ignored = ls-files --others -i --exclude-standard
lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%an, %cr)%Creset' --abbrev-commit --date=relative --topo-order
@mccutchen
mccutchen / .gitconfig
Created June 4, 2015 15:58
Basic ~/.gitconfig
[user]
name = YOUR NAME HERE
email = YOUR EMAIL ADDRESS HERE
[alias]
br = branch
ci = commit
co = checkout
lg = log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%an, %cr)%Creset' --abbrev-commit --date=relative --topo-order
st = status -sb
up = fetch --all --prune
def weighted_choice(choices):
# http://stackoverflow.com/a/3679747/151221
total = sum(w for c, w in choices)
r = random.uniform(0, total)
for c, w in choices:
r -= w
if r <= 0:
return c
#!/bin/bash
if [ -z "$1" ]; then
echo "Usage: `basename $0` BRANCHNAME"
exit 1
fi
git fetch origin
git checkout -b "$1" origin/master --no-track
>>> import urllib
>>> url = 'http://www.newyorker.com/magazine/2014/10/13/cooka%C2%80%C2%99s-tale'
>>> urllib.unquote(url)
'http://www.newyorker.com/magazine/2014/10/13/cooka\xc2\x80\xc2\x99s-tale'
>>> print _
http://www.newyorker.com/magazine/2014/10/13/cooka€™s-tale
>>> urllib.unquote(url).decode('utf8')
u'http://www.newyorker.com/magazine/2014/10/13/cooka\x80\x99s-tale'
>>> print _
http://www.newyorker.com/magazine/2014/10/13/cooka€™s-tale
@mccutchen
mccutchen / gen_batches.py
Last active August 29, 2015 14:07
flexible batching of sequences in Python
def gen_batches(xs, size):
"""
Given a sequence xs and a batch size, yield batches from the sequence as
lists of length size, where the last batch might be smaller than the
rest.
>>> list(gen_batches(range(9), 3))
[[0, 1, 2], [3, 4, 5], [6, 7, 8]]
>>> list(gen_batches(range(11), 3))