Skip to content

Instantly share code, notes, and snippets.

@samklr
samklr / spark-standalone.sh
Last active August 29, 2015 14:16
Spark Standalone Cluster
--------Spark Standalone---------
Prerequistes :
* Set JAVA_HOME env variable
* Configure ssh so master and workers can talk without pw
- $ ssh-keygen -- enter
- Copy the SSH Public Key (id_rsa.pub) to the root account on your target hosts.
.ssh/id_rsa .ssh/id_rsa.pub
- Add the SSH Public Key to the authorized_keys file on your target hosts.
$ cat id_rsa.pub >> authorized_keys
package com.developerb.dropbot;
import com.developerb.dropbot.instrumentation.MethodInvocationTimingInterceptor;
import com.google.inject.AbstractModule;
import com.yammer.metrics.annotation.Timed;
import static com.google.inject.matcher.Matchers.annotatedWith;
import static com.google.inject.matcher.Matchers.any;
/**
sudo ufw allow proto tcp to any port 9200:9400
sudo ufw allow proto tcp to any port 54328
network.host: 127.0.0.1
#!/usr/bin/env bash
# update apt
sudo apt-get update
# install java
sudo apt-get install openjdk-7-jre-headless -y
# install elasticsearch
wget https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.5.0.deb
sudo apt-get -y install build-essential tcl8.5
wget http://download.redis.io/releases/redis-2.8.7.tar.gz
tar -xvf redis-2.8.7.tar.gz
cd redis-2.8.7
cd redis-2.8.7
sudo make install
import scala.collection.immutable.List
import spark.SparkContext._
import spark.streaming._
import spark.streaming.StreamingContext._
import spark.streaming.dstream._
object DataStream extends App {
val reportHeader = """----------------------------------------------
@samklr
samklr / mesos-docker.sh
Created March 31, 2015 15:36
Mesos Docker launcher scripts
#! /bin/bash
HOST_IP=10.11.31.7 ##Put your IP here
# or HOST_IP = ${/sbin/ifconfig eth0 | grep 'inet addr:' | cut -d: -f2 | awk '{ print $1}' }
#Start Zookeeper
docker run -d -p 2181:2181 -p 2888:2888 -p 3888:3888 garland/zookeeper
#Mesos Master
docker run --net="host" \
@samklr
samklr / cassandra-marathon.json
Last active August 29, 2015 14:18
Cassandra Marathon
{
"id": "/ctrlpkw/db",
"apps": [
{
"id": "samklr/risk-metrix",
"apps": [
{
"id": "samklr/cassandra-seed",
"constraints": [
[
  1. Feature Learning
  1. Deep Learning
@samklr
samklr / meetup.sh
Created April 4, 2015 21:37
Curling meetup attendees lists
#!/bin/zsh
#add in your meetup.com API key here
local key
key=
#add in your event ID here
local event
event=