Daniel Truemper truemped


    
    View gist:8259970 
  
  
    File suppressed. Click to show.
  
  
General Background and Overview


Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
Models and Issues in Data Stream Systems

Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet

Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
[Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep


    View tmux_cheatsheet.markdown 
  
  
    File suppressed. Click to show.
  
  
tmux cheatsheet

As configured in my dotfiles.

start new:

tmux


start new with session name:

      
    View list.md 
  
  
    File suppressed. Click to show.
  
  
        A friend asked me for a few pointers to interesting, mostly recent papers on data warehousing and "big data" database systems, and I figured I'd share the list. This is biased and rather incomplete but maybe of interest to someone. While many are obvious choices, I think there are a few underappreciated gems.


Dataflow/Stream Processing Engines:

Dryad--general-purpose distributed parallel dataflow engine

http://research.microsoft.com/en-us/projects/dryad/eurosys07.pdf

Google Dremel--columnar storage for fast queries (c.f. Impala)

http://static.googleusercontent.com/external_content/untrusted_dlcp/research.google.com/en/us/pubs/archive/36632.pdf

      
    View async_mock_example.py 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
          
          
            from mock import CallableMixin, NonCallableMock
 
class AsyncCallableMixin(CallableMixin):
    """Change the __call__ method such that it does not return but call the
    `callback` kwarg with the return value.
    """
 
    def __call__(_mock_self, *args, **kwargs):
        cb = kwargs.get('callback', None)

          
    View setup-statsd.sh 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
            10
          
          
            # install git
sudo apt-get install g++ curl libssl-dev apache2-utils
sudo apt-get install git-core
# download the Node source, compile and install it
git clone https://github.com/joyent/node.git
cd node
./configure
make
sudo make install
# install the Node package manager for later use

          
    View thumbnails.py 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
            10
          
          
            # vim: set fileencoding: utf-8 :
#
import os
 
try:
    from PIL import Image
except ImportError:
    import Image
 
try:

          
    View less2stylus.coffee 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
            10
          
          
            # Quick hack of regular expressions to convert twitter bootstrap from LESS to Stylus
less2stylus = (string) ->
  string = string
    .replace(/^(\ *)(.+)\ +\{\ *\n?\ */mg, "$1$2\n$1  ") # remove opening brackets
    .replace(/^(\ *)([^\ ]+)\ +\{\ *\n?\ *?/mg, "$1$2\n$1  ") # remove opening brackets
    .replace(/\ *\{\ *\n*/g, "\n") # remove opening brackets again (some random cases I'm too lazy to think through)
    .replace(/\ *\}\ *\n*/g, "\n") # remove closing brackets
    .replace(/\;\ *?$/gm, "") # remove semicolons
    .replace(/@(\w+):(\ *)\ /g, "\$$1$2 = ") # replace @variable: with $variable =
    .replace(/\@/g, "\$")

          
    View gist:1717523 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
            10
          
          
            # Set cache dir
proxy_cache_path /var/cache/nginx levels=1:2 
                 keys_zone=microcache:5m max_size=1000m;
 
# Virtualhost/server configuration
server {
    listen   80;
    server_name  yourhost.domain.com;
 
    # Define cached location (may not be whole site)

          
    View ubuntu puppet server passenger install.txt 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
            10
          
          
            Puppet Install steps for Ubuntu 10.04, ruby 1.8, and passenger
 
apt-get -y update
apt-get -y dist-upgrade
reboot
 
############################################
 
apt-get -y install build-essential 
apt-get -y install bsubversion apache2 libcurl4-openssl-dev libssl-dev mysql-server

          
    View sample.config 
  
  
    File suppressed. Click to show.
  
  
            1
            2
            3
            4
            5
            6
            7
            8
            9
            10
          
          
            global
    maxconn     10000 # Total Max Connections. This is dependent on ulimit
    nbproc      2
 
defaults
    mode        http
    option      redispatch
    maxconn     2000
    contimeout  5000
    clitimeout  50000