Skip to content

Instantly share code, notes, and snippets.

View lbradstreet's full-sized avatar

Lucas Bradstreet lbradstreet

View GitHub Profile
(ns onyx.metrics.riemann-playground)
(require '[riemann.client :as r])
(def c (r/tcp-client {:host "192.168.99.100" :port 35002}))
(defn intensive-computation []
(let [prom (r/send-events c (repeat 100000 {:status "ok"}))]
(println "Obtained promise")
(println (deref prom 5000 ::timeout5000))))
Host onyx-kafka
HostName github.com
User git
IdentityFile /home/ubuntu/.ssh/id_onyx-kafka
IdentitiesOnly yes
Host onyx-kafka-0.8
HostName github.com
User git
IdentityFile /home/ubuntu/.ssh/id_onyx-kafka-0.8
IdentitiesOnly yes
(ns onyx.plugin.http-output-gzip-test
(:require [clojure.core.async :refer [go chan >! >!! <!! close!]]
[clojure.test :refer [deftest is]]
[taoensso.timbre :refer [info]]
[onyx.test-helper :refer [with-test-env]]
[onyx.plugin.core-async :refer [take-segments!]]
[onyx.plugin.http-output]
[qbits.jet.server]
[onyx.api])
(:import [java.util.zip GZIPOutputStream]
@lbradstreet
lbradstreet / gist:1a19f7e9943c1864b1dc
Created March 29, 2016 11:23
ABS Peer subscriber # rationale
Three tasks:
A -> B -> C
Two peers on B.
A publishes to stream subscribed by both B peers:
Barrier 1, m1, m2, m3, m4, m5, Barrier 2
B P1 reads:
Barrier 1, m1, m2, m3
@lbradstreet
lbradstreet / gist:a6e389cfa70f4a37675a
Last active March 28, 2016 14:22
Aeron Publications and Subscribers
Two media drivers are launched:
m1 on 192.168.0.1, m2 on 192.168.0.2
One subscriber is added via addSubscription:
Media driver m2 channel "udp://192.168.0.2:42000" stream 1
Three publications are added via addPublication:
Publication 1 on m1 channel "udp://192.168.0.2:42000" stream 1
Media driver m1 creates a publication termBuffer on 192.168.0.1 host
{:valid? true,
:job-invariants
{:job-completed? true,
:reads-correct-jobs? true,
:all-written-read? true},
:cluster-invariants
{:all-peers-up? true,
:peers-match-pulses? true,
:accepting-empty? true,
:prepared-empty? true},
Host onyx-kafka
HostName github.com
User git
IdentityFile /home/ubuntu/.ssh/id_onyx-kafka
IdentitiesOnly yes
Host onyx-datomic
Hostname github.com
User git
IdentityFile /home/ubuntu/.ssh/id_onyx-datomic
IdentitiesOnly yes
(ns test-app-docker-metrics.launcher.launch-prod-peers
(:gen-class)
(:require [aero.core :refer [read-config]] [clojure.core.async :refer [<!! chan]]
[test-app-docker-metrics.jobs.sample-submit-job] [onyx.plugin.kafka] [onyx.plugin.sql]
[taoensso.timbre :as t]
[taoensso.timbre.appenders.3rd-party.rotor :as rotor]
[onyx.lifecycle.metrics.metrics]
[onyx.lifecycle.metrics.timbre]))
(defn standard-out-logger
(defrecord ReadDatoms [task-map unroll db conn datoms-per-segment datoms top-chunk-index top-acked-chunk-index pending-chunk-indices]
PluginReader
(init [this]
(let [conn (safe-connect task-map)
db (safe-as-of task-map conn)]
(assoc this
:db db
:conn conn
:top-chunk-index (atom -1)
:top-acked-chunk-index (atom -1)
diff --git a/test/onyx/plugin/http_output_test.clj b/test/onyx/plugin/http_output_test.clj
index b18178e..e46c7de 100644
--- a/test/onyx/plugin/http_output_test.clj
+++ b/test/onyx/plugin/http_output_test.clj
@@ -12,10 +12,11 @@
[{:url "http://localhost:41300/" :args {:body "a=1" :as :json}}
{:url "http://localhost:41300/" :args {:body "b=2" :as :json}}
{:url "http://localhost:41300/" :args {:body "c=3" :as :json}}
- :done])
+ ;:done