Skip to content

Instantly share code, notes, and snippets.

View timmc's full-sized avatar

Tim McCormack timmc

View GitHub Profile
@timmc
timmc / gist:4a4711c15ff87b2735f2c74bc1efba4a
Last active March 16, 2020 01:17
Download all CDC MMWR for 2017
for i in {01..42}; do curl -sS "https://www.cdc.gov/mmwr/volumes/66/wr/pdfs/mm66${i}.pdf" > "66_${i}.pdf" & done
for i in {43..50}; do curl -sS "https://www.cdc.gov/mmwr/volumes/66/wr/pdfs/mm66${i}-H.pdf" > "66_${i}.pdf" & done
curl -sS "https://www.cdc.gov/mmwr/volumes/66/wr/pdfs/mm665152-H.pdf" > "66_51_52.pdf"
Text extraction:
for x in ./*; do pdftotext $x $x.txt; done
#!/bin/bash
# Call a script in a tight loop from multiple processes.
# Run as `$0 master <#seconds> <#children> script-and-args...`.
# seconds and children control run length and concurrency
# Modes:
# master <seconds> <children>: Fork and run n child processes for m seconds
# child: Call on tight loop, indefinitely
# test: Call just once
#!/bin/bash
cd ~/sync/
function remove_timestamps {
grep -v -Pe '^\+#[0-9]+$' -
}
function baseline_no_ts {
git show HEAD:home/.bash_history | remove_timestamps
#!/bin/bash
# Generates a TOTP code and copies it to the clipboard.
# Accepts path to a GPG-encrypted TOTP secret key file.
# Usage: $0 <encrypted-secrets-file>
#
# Encrypt a secret like this: xsel -bo | gpg2 --encrypt --recipient 0x32D0F478 > ~/.2fa/aws.gpg
set -eu -o pipefail
secretsFile="$1"
@timmc
timmc / sync-setter.kt
Created November 7, 2019 00:51
A synchronized setter with side-effects in Kotlin (probably a bad idea!)
private var writeLock = Object()
/**
* The canonical version of config that is on disk, barring any sneaky
* writes; setting this property causes a disk write.
*/
var config: Config = repo.config.readJson()
set(newConfig) {
synchronized(writeLock) {
repo.config.writeJson(newConfig)
@timmc
timmc / pwned-passwords-sqlite-build.py
Last active March 29, 2023 15:51
Building a sqlite DB for the Pwned Passwords data
#!/usr/bin/env python3
# Build a SQLite3 DB for looking up SHA-1 hashes of leaked passwords.
#
# This can be fed the txt file from one of Have I Been Pwned's hash
# lists available from https://haveibeenpwned.com/Passwords -- but any
# text file with line format ``hash-hex:count`` will work.
#
# When run on the v5 hash-ordered SHA-1 file, expect the build to take
# about 35 minutes and produce a 15.7 GiB file (~30.5 bytes per record).
#
@timmc
timmc / output-2-10.tsv
Created June 5, 2019 22:25
Uptime output for in/out hysteresis healthcheck
0.00 1.00000
0.01 0.99937
0.02 0.99507
0.03 0.98962
0.04 0.98327
0.05 0.96881
0.06 0.95235
0.07 0.94017
0.08 0.91365
0.09 0.88673
@timmc
timmc / elb-hysteresis-2-10.svg
Last active June 5, 2019 19:51
Simulating ELB-style hysteresis for a host that exhibits random failures for both the healthcheck and regular requests
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@timmc
timmc / weighted-shuffle-sampling.clj
Created March 25, 2019 13:11
A sampling-based version of weighted-shuffle (better to use the exponential random solution)
;; This is asymptotically slower (n^2) than the exponential random sort
;; one (n log n) shown in https://gist.github.com/timmc/1211c1ac8ae96c2b42c94124005b5414
;; but it is preserved here for possible later interest
(defn weighted-random-sample
"Given a coll of weights, pick one according to a weighted-random
selection, and return its index. Weights must be non-negative."
[weights]
(when (empty? weights)
(throw (IllegalArgumentException. "Cannot sample from empty list")))
(defn weighted-shuffle
"Perform a weighted shuffle on a collection. weight-fn is called at
most once for every element in the collection."
[weight-fn coll]
(->> coll
(shuffle) ;; tie-break any zero weights
(map (fn [el]
;; Bound the weight to positive values
(let [weight (Math/max Double/MIN_VALUE (double (weight-fn el)))
;; Weighted Random Sampling (2005; Efraimidis, Spirakis)