ikebrown

Backend Architectures

Twitter

ror, scala, jetty, erlang, thrift, mongrel, comet server, my-sql, memchached, varnish, kestrel(mq), starling, gizzard, cassandra, hadoop, vertica, munin, nagios, awstats

• Introduce Gizzard of Twitter - 트위터의 새로운 분산 관리 라이브러리 Gizzard를 소개합니다.
• The Architecture Twitter Uses to Deal with 150M Active Users, 300K QPS, a 22 MB/S Firehose, and Send Tweets in Under 5 Seconds
• Scaling Twitter: Making Twitter 10000 Percent Faster

ikebrown

from django.contrib.sessions.backends.base import SessionBase, CreateError
from django.conf import settings
from django.utils.encoding import force_unicode
import redis

class SessionStore(SessionBase):
    """ Redis store for sessions"""

    def __init__(self, session_key=None):
        self.redis = redis.Redis(

ikebrown

Backend Architectures

ARCHITECTURES

Twitter

ror, scala, jetty, erlang, thrift, mongrel, comet server, my-sql, memchached, varnish, kestrel(mq), starling, gizzard, cassandra, hadoop, vertica, munin, nagios, awstats

ikebrown

Set up Django, nginx and uwsgi

Steps with explanations to set up a server using:

• virtualenv
• Django
• nginx
• uwsgi

ikebrown

#!/bin/bash
### BEGIN INIT INFO
# Provides:          Django EC2 server
# Required-Start:    networking
# Required-Stop:     networking
# Default-Start:  3 5
# Default-Stop:   0 1 2 6
# Short-Description: Django server management script
# Description:       Django server management script
### END INIT INFO

ikebrown

Basic Ubuntu 10.04.3 LTS server setup to deploy Django appplications

This document covers basic steps to setup

• a django user who will serve Django applications,
• [Nginx] as web server,
• [Gunicorn] as a WSGI and
• [PostgreSQL] with [PostGIS] as database server.

...on a default Ubuntu 10.04 server installation.

ikebrown

Here's how to install PostgreSQL and have it run automatically at startup, on an Ubuntu 10.04 virtual machine using Vagrant. This took me a while to figure out:

Add the default lucid32 base box to your vagrant, if you haven't already:

host> vagrant box add lucid32 http://files.vagrantup.com/lucid32.box 

Now make a new lucid32 virtual machine and install postgresql on it:

ikebrown

sudo add-apt-repository ppa:pitti/postgresql
sudo apt-get update
sudo apt-get install postgresql-9.2 postgresql-server-dev-9.2 postgresql-contrib-9.2

sudo su -l postgres
psql -d template1 -p 5433
CREATE EXTENSION IF NOT EXISTS hstore;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
service postgresql stop
/usr/lib/postgresql/9.2/bin/pg_upgrade -b /usr/lib/postgresql/9.1/bin -B /usr/lib/postgresql/9.2/bin -d /var/lib/postgresql/9.1/main/ -D /var/lib/postgresql/9.2/main/ -O "-c config_file=/etc/postgresql/9.2/main/postgresql.conf" -o "-c config_file=/etc/postgresql/9.1/main/postgresql.conf"

ikebrown

*filter

#  Allow all loopback (lo0) traffic and drop all traffic to 127/8 that doesn't use lo0
-A INPUT -i lo -j ACCEPT
-A INPUT -d 127.0.0.0/8 -j REJECT

#  Accept all established inbound connections
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

#  Allow all outbound traffic - you can modify this to only allow certain traffic

ikebrown

Slides:

https://www.dropbox.com/s/2j50hcqb8rk8x0i/fast-track-to-akka-preparations.pdf
https://www.dropbox.com/s/04beouos9bbhw2z/fast-track-to-akka.pdf
https://www.dropbox.com/s/fv8gtqs3p8h1dr6/fast-track-to-akka-part2.pdf

Solutions:
https://www.dropbox.com/s/ocxx0rf0wt07asa/training-akka-solutions.zip

Futures and Promises in Scala 2.10 (a recent talk):