sheepkiller

#System Design Cheatsheet

Picking the right architecture = Picking the right battles + Managing trade-offs

##Basic Steps

1. Clarify and agree on the scope of the system

• User cases (description of sequences of events that, taken together, lead to a system doing something useful)
  • Who is going to use it?
  • How are they going to use it?

sheepkiller

# Docker compose to have Zookeeper, Kafka, and Kafka Manager running for development.

zookeeper:
  image: confluent/zookeeper
  ports:
    - "2181:2181"

kafka:
  # My PR to update to 0.9.0.1 has been already merged, so very soon it will be possible
  # to replace my kafka fork by the original one **wurstmeister/kafka:0.9.0.1**

sheepkiller

- name: Test SQS queue
  hosts: localhost
  gather_facts: no
  vars:
    aws_access_key: XXXXXXXXXXXXXXX
    aws_secret_key: XXXXXXXXX
    aws_region: eu-west-1
    queue_name: test-queue-2
    deadletter_queue_arn: arn:aws:sqs:eu-west-1:YYYYYYYYYY:deadletter-queue
  tasks:

sheepkiller

// A small SSH daemon providing bash sessions
//
// Server:
// cd my/new/dir/
// #generate server keypair
// ssh-keygen -t rsa
// go get -v .
// go run sshd.go
//
// Client:

sheepkiller

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?

sheepkiller

1. General Background and Overview

• Probabilistic Data Structures for Web Analytics and Data Mining : A great overview of the space of probabilistic data structures and how they are used in approximation algorithm implementation.
• Models and Issues in Data Stream Systems
• Philippe Flajolet’s contribution to streaming algorithms : A presentation by Jérémie Lumbroso that visits some of the hostorical perspectives and how it all began with Flajolet
• Approximate Frequency Counts over Data Streams by Gurmeet Singh Manku & Rajeev Motwani : One of the early papers on the subject.
• [Methods for Finding Frequent Items in Data Streams](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.187.9800&rep

sheepkiller

root@dcoker-registry:/root # pkg install -y py27-supervisor libevent2 py27-pip-1.5.6
Updating repository catalogue
FreeBSD repository is up-to-date
All repositories are up-to-date
pkg: mplayerxp has a missing dependency: win32-codecs
The following 10 packages will be affected (of 0 checked):

New packages to be INSTALLED:
        py27-supervisor: 3.0_1,1
        py27-meld3: 0.6.8_1