prasincs

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?

prasincs

apache_dl(){
    wget $(curl -L "$1?as_json=1" | python -c 'import sys, json; i =json.load(sys.stdin); print i["preferred"]+i["path_info"].strip()')
}

prasincs

;; -*- mode: Clojure; -*-

                                        ;(load-from-resource "/Users/pgautam/Projects/riemann-acknowledgement/target/riemann-acknowledgement-0.1.3.jar")
(include "../include")
(load-plugins)

(instrumentation {:interval 5
                  :enabled? false})

(let [host "0.0.0.0"]

prasincs

(ns prime-days
  (:require [clj-time.core :as t]
            [clj-time.format :as f]
            [clojure.pprint :refer [pprint]]))

(defn prime? [n]
  (.isProbablePrime (BigInteger/valueOf n) 1))

(defn formatted-date [n]
  (f/unparse

prasincs

For a tmux status line as seen in the example image for the wemux project:

The session on the left in the example screen shot uses a patched font from the vim-powerline project. Inconsolata-dz, you beautiful creature.

To duplicate the left status line add the following lines to your ~/tmux.conf

set -g status-left-length 32
set -g status-right-length 150

prasincs

#!/bin/bash
set +x
set -o pipefail
spark_tmp_dirs=$(sudo ls -d /tmp/spark-*)
for dir in $spark_tmp_dirs; do
	num_procs_using_dir=$(sudo lsof +D $dir | wc -l)
	if [ "$num_procs_using_dir" = "0" ]; then
		sizeof_dir=$(sudo du -sh . | cut -d$'\t' -f 1)
		echo "Deleting $dir -- should free up $sizeof_dir"
		sudo rm -r $dir

prasincs

#!/bin/bash
function remove_bucket() {
    s3cmd rm --force --recursive s3://$1 && s3cmd rb s3://$1
}

dry_run=0
HBASE_SNAPSHOT_BUCKETS=$(aws s3api list-buckets | jq -r ".Buckets[].Name" | grep "hbase-snapshots")
echo "About to delete the following buckets: $HBASE_SNAPSHOT_BUCKETS"
for bucket in $HBASE_SNAPSHOT_BUCKETS; do
    echo "Checking $bucket"

prasincs

UNKNOWN = ""
def get_in(d, key, no_key_value):
    if type(d) != dict:
        return d
    else:
        if key[0] in d:
            return get_in(d[key[0]], key[1:], no_key_value)
        else:
            return no_key_value

prasincs

• Finding the size in MB

function s3_dir_size {
  s3cmd du $1 | cut -d' ' -f 1 | { read a; echo $a "/ (1024 * 1024 ) "; } | bc -l | { read a; echo "$a M $1"; }
}

prasincs

Keybase proof

I hereby claim:

• I am prasincs on github.
• I am prasincs (https://keybase.io/prasincs) on keybase.
• I have a public key whose fingerprint is 96FF 054B D808 30CB 7A3E 14DC A269 D112 7D8E ABF8

To claim this, I am signing this object: