Skip to content

Instantly share code, notes, and snippets.

Tim McCormack timmc

Block or report user

Report or block timmc

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
timmc / gist:df5fbb6e069fb8c1c4e181a29930ace3
Last active Aug 9, 2019
Building a sqlite DB for the Pwned Passwords data
View gist:df5fbb6e069fb8c1c4e181a29930ace3

Last executed 2019-06-25 with the v4 dump:

  1. Make sure you have 60 GB free disk space and some extra to spare. Alternatively, take a walk on the wild side and delete source files as soon as you've used them.
  2. Download the SHA-1 (ordered by hash) torrent from
  3. Unpack and strip off the counts:
    7z x -so pwned-passwords-sha1-ordered-by-hash-v4.7z pwned-passwords-sha1-ordered-by-hash-v4.txt | sed 's/:.*//' > hashes.lst
timmc / output-2-10.tsv
Created Jun 5, 2019
Uptime output for in/out hysteresis healthcheck
View output-2-10.tsv
0.00 1.00000
0.01 0.99937
0.02 0.99507
0.03 0.98962
0.04 0.98327
0.05 0.96881
0.06 0.95235
0.07 0.94017
0.08 0.91365
0.09 0.88673
timmc / elb-hysteresis-2-10.svg
Last active Jun 5, 2019
Simulating ELB-style hysteresis for a host that exhibits random failures for both the healthcheck and regular requests
View elb-hysteresis-2-10.svg
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
timmc / weighted-shuffle-sampling.clj
Created Mar 25, 2019
A sampling-based version of weighted-shuffle (better to use the exponential random solution)
View weighted-shuffle-sampling.clj
;; This is asymptotically slower (n^2) than the exponential random sort
;; one (n log n) shown in
;; but it is preserved here for possible later interest
(defn weighted-random-sample
"Given a coll of weights, pick one according to a weighted-random
selection, and return its index. Weights must be non-negative."
(when (empty? weights)
(throw (IllegalArgumentException. "Cannot sample from empty list")))
View weighted-shuffle-fancy.clj
(defn weighted-shuffle
"Perform a weighted shuffle on a collection. weight-fn is called at
most once for every element in the collection."
[weight-fn coll]
(->> coll
(shuffle) ;; tie-break any zero weights
(map (fn [el]
;; Bound the weight to positive values
(let [weight (Math/max Double/MIN_VALUE (double (weight-fn el)))
;; Weighted Random Sampling (2005; Efraimidis, Spirakis)
timmc /
Last active Mar 19, 2019
Success-prioritized, concurrency-limited stochastic fallback cascade


  • State
    • For each node, store:
      • Rolling window of 6 historical stat buckets
      • One additional "sticky" bucket
    • Write stats to newest bucket
    • Age out the oldest bucket every 5 seconds, and add a new one
      • If the oldest bucket had data, copy it over the "sticky" bucket
    • Recorded stats: Number of finished requests, number that
while read line; do
url_encoded="${line//+/ }"
printf '%b\n' "${url_encoded//%/\\x}"
timmc /
Created Jan 22, 2019
Custom query term splitter, as state machine (demonstration only; a regex with possessive quantifiers slightly out-performs this)
public static List<String> splitTermsStateMachine(String query) {
// Horizontal or vertical whitespace
Pattern isWhitespace = Pattern.compile("[\\h\\v]");
//== Manually managed state bits ==//
// If non-null, we're in a term, and this contains the term so far
StringBuilder currentTerm = null;
// True iff inside a quoted run
boolean inQuotes = false;
View MapEntry.kt
data class MapEntry<K: Any?, V: Any?>(
override val key: K,
override val value: V
): Map.Entry<K, V>
timmc / layout.kt
Created Nov 18, 2018
Wanted: Fluent interface for specifying filesystem layout
View layout.kt
class Repo : Layout() {
val config by leaf("config.json")
val myKeyring by leaf("self.keyring")
val posts by dir("posts", ::PostsDir)
class PostsDir : Layout() {
val post by multiple().dir(name = """[0-9]+""".toRegex(), ::OnePost)
You can’t perform that action at this time.