gleicon

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?

gleicon

import re
import boto3
import csv
from botocore.exceptions import ClientError

ec2 = boto3.client('ec2')

def get_snapshots():
    return ec2.describe_snapshots(OwnerIds=['self'])['Snapshots']

gleicon

# Helper class for statistics
# see: http://www.johndcook.com/blog/standard_deviation/
class Stats:
    """ Uses Welford's method to calculate stats.
        Assumes positive values.
        It's not thread safe

        stats = Stats("ConnectionTimeStats")
        stats.add(0.223)
        stats.add(1.343)

gleicon

Relacionamento de quem fez X, também fez X

O objetivo aqui é pegar uma entrada composta por um csv com: produto, sessão. E considerando que a ação X foi aplicada a ele criar um cara que pega as relações por sessão.

Exemplo de entrada:

1231312124,cb808cb9-3bfd-4c0c-ad20-2e181b9b3a93
1231212124,cb808cb9-3bfd-4c0c-ad20-2e181b9b3a93
1231112124,cb808cb9-3bfd-4c0c-ad20-2e181b9b3a93
1231762124,cb808cb9-3bfd-4c0c-ad20-2e181b9b3a93

gleicon

from tornado import auth, httpclient, httputil, escape
import urllib
import functools
 
class LinkedInMixin(auth.OAuth2Mixin):
    """
    LinkedIn authentication using OAuth2.
 
    Example usage::
 

gleicon

upstream backend {
    server localhost:8080;
    #server backup1.example.com:8080   backup;
    #server backup2.example.com:8080   backup;
}

# Set cache dir
proxy_cache_path /var/cache/nginx levels=1:2 keys_zone=one:10m;

# Set cache key to include identifying components

gleicon

Latency Comparison Numbers
--------------------------
L1 cache reference                            0.5 ns
Branch mispredict                             5   ns
L2 cache reference                            7   ns             14x L1 cache
Mutex lock/unlock                            25   ns
Main memory reference                       100   ns             20x L2 cache, 200x L1 cache
Compress 1K bytes with Zippy              3,000   ns
Send 1K bytes over 1 Gbps network        10,000   ns    0.01 ms
Read 4K randomly from SSD*              150,000   ns    0.15 ms

gleicon

// Go and Server Sent Events for HTTP/1.1 and HTTP/2.0
//go:generate go run $GOROOT/src/crypto/tls/generate_cert.go -host localhost

package main

import (
	"fmt"
	"io"
	"log"
	"net/http"

gleicon

import math
import random

adjs=["autumn", "hidden", "bitter", "misty", "silent", "empty", "dry", "dark",
    "summer", "icy", "delicate", "quiet", "white", "cool", "spring", "winter",
    "patient", "twilight", "dawn", "crimson", "wispy", "weathered", "blue",
    "billowing", "broken", "cold", "damp", "falling", "frosty", "green",
    "long", "late", "lingering", "bold", "little", "morning", "muddy", "old",
    "red", "rough", "still", "small", "sparkling", "throbbing", "shy",
    "wandering", "withered", "wild", "black", "young", "holy", "solitary",

gleicon

import cyclone.web
import cyclone.auth
from cyclone import escape, httpclient
from cyclone.options import parse_command_line, define, options
from twisted.python import log
from twisted.internet import task, defer, reactor
import sys

try:
    import urllib.parse as urllib_parse  # py3