kgriffs

raise MyError, e.args, sys.exc_info()[2]

kgriffs

N = 5
V = 0.001

for i in range(N):
    print V * (2 ** i)

kgriffs

Appendix D: Conflict or Complimentary?

Web Frameworks

• Django — Flask
• Rails — Sinatra
• Hapi — Express
• Pecan — Falcon

Libraries

kgriffs

# solom/__init__.py
THREAD_LOCAL = threading.local()


# In some pecan helper module...
def before_hook(self):
    # Use thread-local storage so we don't have to 
    # add two extra params to virtuall every single 
    # function in our app!
    solum.THREAD_LOCAL.ctx = context.RequestContext(...)

kgriffs

from __future__ import division

import multiprocessing as mp
import argparse
import random
import time

from gevent import monkey as curious_george
curious_george.patch_all(thread=False, select=False)

kgriffs

Using timestamps as IDs

Helps scale writes:

• For a single master, means don't have to query to find last inserted ID.
• Collisions can be practically impossible
• Avoids having to retry using an optimistic algorithm, in the case that a parallel request beat me to the punch and I have a collision of IDs.

Caveats:

• Timestamp must be granular enough to be unique for expected max message insertion rate per queue. For example, given a goal of 100,000 writes/sec to a specific queue, to avoid collisions the timestamps would need to be stored with at least 100 ms granularity. However, one must also allow for outliers, where occasionally the stars will align and several requests will arrive and request timestamps at the same moment. [wWth that in mind, what is a the practical granularity required? microseconds? 100-nanoseconds?]
• Requires specialized hardware to avoid significant drift. However, NTP may be "good enough" if ~1 ms drift is acceptable.

kgriffs

░░░░░░░░░▄░░░░░░░░░░░░░░▄░░░░
░░░░░░░░▌▒█░░░░░░░░░░░▄▀▒▌░░░
░░░░░░░░▌▒▒█░░░░░░░░▄▀▒▒▒▐░░░
░░░░░░░▐▄▀▒▒▀▀▀▀▄▄▄▀▒▒▒▒▒▐░░░
░░░░░▄▄▀▒░▒▒▒▒▒▒▒▒▒█▒▒▄█▒▐░░░
░░░▄▀▒▒▒░░░▒▒▒░░░▒▒▒▀██▀▒▌░░░ 
░░▐▒▒▒▄▄▒▒▒▒░░░▒▒▒▒▒▒▒▀▄▒▒▌░░
░░▌░░▌█▀▒▒▒▒▒▄▀█▄▒▒▒▒▒▒▒█▒▐░░
░▐░░░▒▒▒▒▒▒▒▒▌██▀▒▒░░░▒▒▒▀▄▌░
░▌░▒▄██▄▒▒▒▒▒▒▒▒▒░░░░░░▒▒▒▒▌░

kgriffs

$ pbpaste | python -m json.tool | pbcopy

kgriffs

Python performance testing tools

• http://jiffyclub.github.io/snakeviz/
• http://www.vrplumber.com/programming/runsnakerun/
• http://pythonhosted.org//line_profiler/
• https://docs.python.org/3.4/library/tracemalloc.html
• https://pypi.python.org/pypi/memory_profiler
• https://github.com/jtushman/memory_utils

kgriffs

def parse_host(host, default_port=None):
    # NOTE(kgriff): The value from the Host header may
    # contain a port, so check that and strip it if
    # necessary. This is complicated by the fact that
    # a hostname may be specified either as an IP address
    # or as a domain name, and in the case of IPv6 there
    # may be multiple colons in the string.

    if host.startswith('['):
        # IPv6 address with a port