guenter

Links

• This script: https://gist.github.com/guenter
• Launch a Mesos cluster on Google Compute: https://google.mesosphere.io
• Marathon docs: https://mesosphere.github.io/marathon/
• Marathon 0.7.0-RC2: http://downloads.mesosphere.io/marathon/v0.7.0-RC2/marathon-0.7.0-RC2.tgz

Shared Cluster

Please prefix apps with your name if you use the shared cluster

PatrickCallaghan

The default is to put this on its own node so you will need to start DSE with dse cassandra -k to create a spark analytics node.

First run the https://github.com/PatrickCallaghan/datastax-userinteractions-demo project to populate the Cassandra cluster (follow instructions in README). Use this project to populate the Cassandra db with hundreds of thousands user interactions. The idea is to have users interacting with multiple apps and we can model this by user in Cassandra.

We have an existing table that has all the data for user interactions with certain applications on the appropriate date. Now, for some other requirements, we need the unique users that visited a certain page within an app on a certain day.

So the first requirement was - show me the user interactions with a certain app

Now, we have a new requirement - show me all the users that interacted with a certain app on a particular day.

mesosphere.sh

#! /bin/bash

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv E56151BF
DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
CODENAME=$(lsb_release -cs)
echo "deb http://repos.mesosphere.io/${DISTRO} ${CODENAME} main" | sudo tee /etc/apt/sources.list.d/mesosphere.list

sudo apt-get -y update --fix-missing
sudo apt-get -y install mesosphere

ezhulenev

import scala.collection.immutable.TreeMap

trait ToTreeMap[A] {
  type Result

  def treeMap(x: A): Result
}

trait LowerPriorityToTreeMap {
  implicit def plainMap[K, V](implicit ord: Ordering[K]): ToTreeMap[Map[K, V]] =

miguno

package com.twitter.algebird.caliper

import com.google.caliper.{ Param, SimpleBenchmark }
import com.google.common.hash.{ HashFunction, Hashing }

/**
 * Benchmarks the hashing algorithms used by Count-Min sketch for CMS[BigInt].
 *
 * The input values are generated ahead of time to ensure that each trial uses the same input (and that the RNG is not
 * influencing the runtime of the trials).

takei-shg

import scalaz._
import Scalaz._
import scalaz.OptionT._
import com.twitter.util.Future

/**
 * Simple example of Future chain, which only shows type level consistency.
 * Future chain can be implemented with optionT or simple flatMap chain.
 */
object Main extends App {

3h4x

{
  "id": "/ctrlpkw",
  "groups": [
    {
      "id": "/ctrlpkw/db",
      "apps": [
          {
              "id": "/ctrlpkw/db/cassandra-seed",
              "constraints": [["hostname", "UNIQUE"]],
              "ports": [7199, 7000, 7001, 9160, 9042],

samklr

Internet Scale Services Checklist

A checklist for designing and developing internet scale services, inspired by James Hamilton's 2007 paper "On Desgining and Deploying Internet-Scale Services."

• http://mvdirona.com/jrh/talksandpapers/jamesrh_lisa.pdf

Basic tenets

• Does the design expect failures to happen regularly and handle them gracefully?
• Have we kept things as simple as possible?

philipithomas

using JuMP
using Cbc

#=
We pass in the variable "pools" in this format that goes through a separate
pre-processing script that pipes JSON to a Julia JSON loader.


{
    "awesome-pool-prod": {

aphyr

#!/usr/bin/env ruby
require 'sequel'
require 'fileutils'
require 'uri'
require 'pp'

def home
  ENV['HOME']
end