Skip to content

Instantly share code, notes, and snippets.

boiling the ocean

Zach Tellman ztellman

boiling the ocean
View GitHub Profile
View advent2020-17.clj
(ns advent17.core
[ :as io]
[clojure.string :as str]))
(defn neighbors [[x y z w]]
(for [x' (range (dec x) (+ 2 x))
y' (range (dec y) (+ 2 y))
z' (range (dec z) (+ 2 z))
View boolean.js
var text = new PointText({
position: + [0, 200],
fillColor: 'black',
justification: 'center',
fontSize: 20
var originals = new Group({ insert: false }); // Don't insert in DOM.
var square = new Path.Circle({
View quasi-quote-macro.clj
(defmacro if-in-str [haystack & clauses]
(fn [clause]
(let [needle (first clause)
is-present-form (second clause)
is-not-present-form (nth clause 2 nil)]
`(if (> (.indexOf ~haystack ~needle) -1)
View kafka-manifold.clj
(defn kafka-sink []
(let [s (s/stream)]
(fn [msg]
(d/future (synchronous-put-to-kafka msg))))

This relates to work I'm doing on a system for distributed outlier detection, as described here.

In this system, the count-min sketch is used as a distributed filter. Over some interval, the CMS is populated on all the edge nodes, sent to a server for merging, and the merged CMS is broadcast back out to the edges. On the following interval, each new key is run through the "global" CMS, and if it's over some threshold, it is added to the set of potential outliers, which are precisely counted.

This is, as described so far, an adaptation of this 2003 paper from single-node to multi-node. However, that paper assumes that the threshold is known ahead of time.

In most cases, though, what constitutes an "outlier" is a moving target. We'd like to adapt as the distribution changes. Previously, I was using a consistent uniform sample of the keys to identify the threshold, but we'd have to take a very

View async.clj
(require '[manifold.deferred :as d])
(defn wrap-async-handler [f]
(fn [req]
(let [rsp (d/deferred)]
(f req #(d/success! rsp %) #(d/error! rsp %))
public final class Netty {
private Netty() {
private static final Field QUEUE;
private static final Class BITS;
private static final Class VM;
ztellman / cljx.clj
Last active November 20, 2015 22:24
View cljx.clj
(defn- queue []
#+clj clojure.lang.PersistentQueue/EMPTY
#+cljs cljs.core/PersistentQueue.EMPTY)
(defn matching-inputs
"Returns a lazy sequence of input sequences which the automaton will match."
(let [fsm (-> fsm ->dfa final-minimize)
accept? (set (accept fsm))
q (atom (conj (queue) [(start fsm) []]))]
View cmsketch.lua
local function delta(t0, t1)
return ((t1[1] - t0[1]) * 1000000) + (t1[2] - t0[2])
local t0,t1
local s1 ="get", KEYS[1])
local s2 = ARGV[1]
if s1 == false then"set", KEYS[1], s2)
package compression;
import java.lang.Exception;
import java.nio.ByteBuffer;
import java.nio.DoubleBuffer;
public class FPC {
// adapted from
private static final int PREDICT_TABLE_SIZE = 32768;