mping

1. General Background and Overview

• Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
• Models and Issues in Data Stream Systems
• Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
• Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
• [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

mping

#
# Wide-open CORS config for nginx
#
location / {

     if ($request_method = 'OPTIONS') {

        add_header 'Access-Control-Allow-Origin' 'http://10.140.10.40';
        
        #

mping

#!/bin/sh
# deploy.sh
N="`readlink \"$1\"`"
mv -T "$1.stage" "$1"
ln -s "$N" "$1.stage"
rm -rf "$N"
cp -aH "$1" "$N"

mping

Latency Comparison Numbers
--------------------------
L1 cache reference                            0.5 ns
Branch mispredict                             5   ns
L2 cache reference                            7   ns             14x L1 cache
Mutex lock/unlock                            25   ns
Main memory reference                       100   ns             20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy              3,000   ns
Send 1K bytes over 1 Gbps network        10,000   ns    0.01 ms
Read 4K randomly from SSD*              150,000   ns    0.15 ms

mping

(def ^:dynamic *indent* 0)
(alter-var-root
 #'clojure.core/load
 (fn [orig]
   (fn [& paths]
     (let [t (System/nanoTime)
           r (binding [*indent* (inc *indent*)]
               (apply orig paths))]
       (binding [*out* *err*]
         (println (apply str (repeat *indent* " ")) (/ (- (System/nanoTime) t) 1000000.0)  paths)

mping

Socks Proxy

Add Remote Host

Add Jstad Connection

mping

Prereq:

apt-get install zsh
apt-get install git-core

installation

Getting zsh to work in ubuntu is weird, since sh does not understand the source command. So, you do this to install zsh

mping

;;  1) first goto https://console.developers.google.com/start/api?id=gmail
;;  -- to turn on gmail api for your account
;;  2) then create a oauth consent with app name
;;  3) then add oauth clientid and download to ./clientsecret.json 
;;  4) boot run
;;  -- when running first time a browser will open to accept authorizaton

(set-env! :dependencies '[[com.google.apis/google-api-services-gmail "v1-rev34-1.21.0"]
                          [com.google.api-client/google-api-client "1.20.0"]
                          [com.google.oauth-client/google-oauth-client-jetty "1.20.0"]])

mping

import scala.util.parsing.combinator._
import scala.language.postfixOps

object Main {
  def main(args:Array[String]) = {
    val sc = new java.util.Scanner(System.in)

    val input = collection.mutable.ListBuffer[String]()
    while(sc.hasNextLine) {
    	input += sc.nextLine

mping

Quick Tips for Fast Code on the JVM

I was talking to a coworker recently about general techniques that almost always form the core of any effort to write very fast, down-to-the-metal hot path code on the JVM, and they pointed out that there really isn't a particularly good place to go for this information. It occurred to me that, really, I had more or less picked up all of it by word of mouth and experience, and there just aren't any good reference sources on the topic. So… here's my word of mouth.

This is by no means a comprehensive gist. It's also important to understand that the techniques that I outline in here are not 100% absolute either. Performance on the JVM is an incredibly complicated subject, and while there are rules that almost always hold true, the "almost" remains very salient. Also, for many or even most applications, there will be other techniques that I'm not mentioning which will have a greater impact. JMH, Java Flight Recorder, and a good profiler are your very best friend! Mea