Tuning Storm+Trident
Tuning a dataflow system is easy:
The First Rule of Dataflow Tuning: * Ensure each stage is always ready to accept records, and * Deliver each processed record promptly to its destination
;; make sure you've set your default project with: | |
;; gcloud config set project <project-name> | |
(require 'tramp) | |
(add-to-list 'tramp-methods | |
'("gcssh" | |
(tramp-login-program "gcloud compute ssh") | |
(tramp-login-args (("%h"))) | |
(tramp-async-args (("-q"))) | |
(tramp-remote-shell "/bin/sh") |
(ns example.nrepl | |
(:require [clojure.tools.nrepl.server :refer [start-server stop-server]] | |
[cider.nrepl :refer [cider-nrepl-handler]]) | |
(:gen-class)) | |
(defn -main [port] | |
(let [port (Integer. port)] | |
(defonce server (start-server :port port :handler cider-nrepl-handler)))) |
Tuning a dataflow system is easy:
The First Rule of Dataflow Tuning: * Ensure each stage is always ready to accept records, and * Deliver each processed record promptly to its destination
#!/usr/bin/ruby | |
# Create display override file to force Mac OS X to use RGB mode for Display | |
# see http://embdev.net/topic/284710 | |
require 'base64' | |
data=`ioreg -l -d0 -w 0 -r -c AppleDisplay` | |
edids=data.scan(/IODisplayEDID.*?<([a-z0-9]+)>/i).flatten | |
vendorids=data.scan(/DisplayVendorID.*?([0-9]+)/i).flatten |
I hereby claim:
To claim this, I am signing this object:
#!/bin/sh | |
TMPCP=/tmp/badcp.txt | |
lein cp | tr ':' '\n' > $TMPCP | |
while read line; do | |
find "$line" -name "*.jar" -exec sh -c 'jar -tf {}| grep -H --label {} '$1'' \; | |
done < "$TMPCP" |
(defn ambrose?- | |
[& bindings] | |
(let [[name bindings] (flow/parse-exec-args bindings) | |
bindings (mapcat (partial apply normalize-sink-connection) | |
(partition 2 bindings)) | |
flow (-> (apply compile-flow name bindings) | |
flow/flow-def | |
flow/compile-hadoop) | |
server (EmbeddedAmbroseCascadingNotifier.)] | |
(.addListener flow server) |
(deftheme bubbleberry "bubbleberry") | |
;; Based on the theme used for LightTable (see: http://www.chris-granger.com/images/lightable/main.png ) | |
(custom-theme-set-variables | |
'bubbleberry | |
'(linum-format " %7i ") | |
'(fringe-mode 5 nil (fringe)) | |
'(powerline-color1 "#3d3d68") | |
'(powerline-color2 "#292945") |
[thread 140515116697344 also had an error]# | |
[thread 140515116697344 also had an error][thread 140515116697344 also had an error][thread 140515116697344 also had an error] | |
# A fatal error has been detected by the Java Runtime Environment: | |
# | |
# SIGSEGV[thread 140515130382080 also had an error][thread 140515130382080 also had an error][thread 140515130382080 also had an error] | |
(0xb) |
(comment | |
(defrel edge a b) | |
(def graph [{:id :gen :in [] :out ["a" "b"]} | |
{:id :plus :in ["a"] :out ["plus"]} | |
{:id :times :in ["plus"] :out ["times"]}]) | |
(fact edge '{:id :gen :in [] :out ["a" "b"]} '{:id :plus :in ["a"] :out ["plus"]}) | |
(fact edge '{:id :plus :in ["a"] :out ["plus"]} '{:id :times in ["plus"] :out ["times"]}) |