Skip to content

Instantly share code, notes, and snippets.

@hadronzoo
hadronzoo / Cargo.toml
Last active November 19, 2019 04:08
SMT microbenchmark in Rust
[package]
name = "triggering"
version = "0.1.0"
authors = ["Joshua Griffith <contact@joshuagriffith.com>"]
edition = "2018"
[dependencies]
rand = "0.7"
[dev-dependencies]
@hadronzoo
hadronzoo / jbg_resume.md
Last active November 12, 2017 22:08
Resume

Joshua Griffith

Experience

2/2014–present Software Developer, Campus Labs (formerly OrgSync), Dallas, TX

  • Created a general-purpose join service using Apache Flink and Scala to extract, filter, denormalize, sort, and summarize structured data for Elasticsearch indexing, which led to an over 50% improvement in data freshness for the Campus Labs Insight product. Typical queries processed hundreds of millions of source rows from over one-hundred tables across several databases multiple times per day. Service runs on Kubernetes and provides a GraphQL-inspired API for describing the relationships between data sources and the desired shape of the results. Supports automatic RDBMS schema discovery, parallel extraction, throttling, Vault-based credential management, and recursive joins using delta-iteration.

  • Built a scheduled ETL process using Flink and Scala to transform and load over 1 billion rows daily into a Data Vault-based data warehouse. Implemented keyset table slicing to parallelize extracti

@hadronzoo
hadronzoo / flink-statsd-mapping.conf
Created May 11, 2017 21:09
statsd-exporter configuration for exporting Flink statsd metrics to Prometheus
@hadronzoo
hadronzoo / watermark-example.clj
Last active November 4, 2016 15:48
Emit watermark changes — useful for keeping track of Kafka partition log offsets
(let [partition (juxt :topic :partition)
raise? #(case (:operation %)
:fetch false
:ack true
(throw (ex-info "unknown operation" %)))
level :offset
operations [{:topic "test" :partition 0 :operation :fetch :offset 0}
{:topic "test" :partition 0 :operation :fetch :offset 1}
{:topic "test" :partition 0 :operation :ack :offset 1}
{:topic "test" :partition 1 :operation :fetch :offset 0}
@hadronzoo
hadronzoo / contiguous-xf.clj
Created November 2, 2016 17:59
Partition contiguous groups of integers
(ns xf.contiguous
(:require [clojure.data.int-map :as i]))
(defn partition-contiguous
([]
(let [buffer (volatile! (i/dense-int-set))]
(fn [xf]
(fn
([] (xf))
([result]
@hadronzoo
hadronzoo / example.log
Last active April 2, 2016 15:54
Generator Example
g = generator(0)
=> [Function]
g()
=> 0
g()
=> 1
g()
=> 2
@hadronzoo
hadronzoo / arity.clj
Last active August 29, 2015 14:14
No matching arities
(ns tctest.core
(:require [clojure.core.typed :as t]))
(t/defprotocol AProtocol
(some-method [this] :- t/AnyInteger
[this arg :- t/Any] :- t/AnyInteger))
(t/ann-datatype AType [])
(deftype AType []
AProtocol
@hadronzoo
hadronzoo / jvm_cursor.clj
Created November 5, 2014 23:03
Clojure MapCursor
(ns cursors.core
(:refer-clojure :exclude [swap! reset!]))
(defprotocol ICursor
(-path [c])
(-state [c]))
(defprotocol IToCursor
(-to-cursor [c state path]))
@hadronzoo
hadronzoo / core.clj
Last active August 29, 2015 14:02
core.typed.async error
(ns test.core
(:require [clojure.core.async :refer [<! >!]]
[clojure.core.typed :refer [ann U]]
[clojure.core.typed.async :refer [Chan go>]]))
(ann test-go [(Chan Integer) -> (U (Chan Integer) nil)])
(defn test-go [in] (go> (<! in)))
(defn minmax-by-codepoint [s]
(let [sorted (reduce (fn [r [c cnt]]
(update-in r [cnt] #((fnil conj (sorted-set)) % c)))
(sorted-map)
(frequencies s))
get-char (juxt (comp first second) first)]
[(get-char (first sorted))
(get-char (last sorted))]))