abhijitiitr

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?

abhijitiitr

Latency numbers every programmer should know

L1 cache reference ......................... 0.5 ns
Branch mispredict ............................ 5 ns
L2 cache reference ........................... 7 ns
Mutex lock/unlock ........................... 25 ns
Main memory reference ...................... 100 ns             
Compress 1K bytes with Zippy ............. 3,000 ns  =   3 µs
Send 2K bytes over 1 Gbps network ....... 20,000 ns  =  20 µs
SSD random read ........................ 150,000 ns  = 150 µs

Read 1 MB sequentially from memory ..... 250,000 ns = 250 µs

abhijitiitr

1. General Background and Overview

• Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
• Models and Issues in Data Stream Systems
• Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
• Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
• [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

abhijitiitr

import java.io.IOException;
import java.net.URLClassLoader;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;

/**
 * Example demonstrating a ClassLoader leak.
 *
 * <p>To see it in action, copy this file to a temp directory somewhere,

abhijitiitr

This note explains how to build Postgres from source and setup to debug it using LLDB on a Mac. I used this technique to research this article:
http://patshaughnessy.net/2014/10/13/following-a-select-statement-through-postgres-internals

1. Shut down existing postgres if necessary - you don’t want to mess up your existing DB or work :)

$ ps aux | grep postgres
pat               456   0.0  0.0  2503812    828   ??  Ss   Sun10AM   0:11.59 postgres: stats collector process       
pat               455   0.0  0.0  2649692   2536   ??  Ss   Sun10AM   0:05.00 postgres: autovacuum launcher process       
pat               454   0.0  0.0  2640476    304   ??  Ss   Sun10AM   0:00.74 postgres: wal writer process       
pat               453   0.0  0.0  2640476    336   ??  Ss   Sun10AM   0:00.76 postgres: writer process       

abhijitiitr

## Functional Track talks from NDC London 2014

### Wednesday 2014-12-3

* [F-Words - Functional Programming Terms With More Than Four Letters - Calvin Bottoms](http://www.ndcvideos.com/#/app/video/2191)
* [Reactive Game Development For The Discerning Hipster - Bodil Stokke](http://www.ndcvideos.com/#/app/video/2221)
* [Erlang Patterns Matching Business Needs -- Torben Hoffman](http://www.ndcvideos.com/#/app/video/2211)
* [Equivalence Classes, xUnit.net, FsCheck, Property-Based Testing -- Mark Seemann](http://www.ndcvideos.com/#/app/video/2291)
* [Functional programming design patterns -- Scott Wlaschin](http://www.ndcvideos.com/#/app/video/2311)
* [Write Your Own Compiler in 24 Hours -- Phillip Trelford](http://www.ndcvideos.com/#/app/video/2281)

abhijitiitr

Generate a new Elixir project using mix and add cowboy and plug as dependencies in mix.exs:

  defp deps do
    [
      {:cowboy, "~> 1.0.0"},
      {:plug, "~> 0.8.1"}
    ]
  end

abhijitiitr

Channels Are Not Enough

... or Why Pipelining Is Not That Easy

Golang Concurrency Patterns for brave and smart.

By @kachayev

Intro

abhijitiitr

% c(spawner).
% c(conc_nif_caller).
% c(niftest).
% spawn(spawner,spawn_multiple_nif_callers,[10]).
-module(spawner).
-export([spawn_multiple_nif_callers/1]).

spawn_multiple_nif_callers(Val) ->
	FinalPidList = lists:foldl(fun(Int_Pid, PidList) -> 
								  Pid = spawn(conc_nif_caller, call_nif, []),

abhijitiitr

-module(conc_nif_caller).
-export([call_nif/0]).

call_nif() ->
	receive
		Int_Val ->
			niftest:hello(Int_Val),
			ok
	end.