Skip to content

Instantly share code, notes, and snippets.

View thinkerbot's full-sized avatar

Simon Chiang thinkerbot

View GitHub Profile
thinkerbot /
Created November 24, 2016 20:24
Google Uploader
# Google Uploader looks at a file timestamp not the video metadata. Run
# this to get the right timestamp to appear on the video. Requires
# exiftool.!topic/photos/oj96JZK14Fs
# brew install exiftool
# ./
ruby -rtime -ryaml -rshellwords -e '
thinkerbot /
Last active October 29, 2016 03:47
Stream compress, encrypt, decrypt, uncompress
time (
ruby -e "STDOUT.sync=true; 100000.times {|i| puts(i.to_s + ' knowledge is power') }" |
gzip --stdout |
openssl enc -aes-128-cfb -pass pass:test -bufsize 256 |
openssl enc -base64 -bufsize 256 |
openssl enc -base64 -d -bufsize 256 |
thinkerbot / sign_requests_with_instance_profile_for_aws_es.rb
Created August 4, 2016 20:03
Sign requests to get access to a role-controlled AWS elasticsearch
require 'aws-sdk'
require 'faraday_middleware'
require 'faraday_middleware/aws_signers_v4'
require 'pp'
conn = 'http://ENDPOINT') do |faraday|
faraday.request :aws_signers_v4,
service_name: 'es',
region: 'REGION'
x, y = 0, 0
xmin, xmax, ymin, ymax = 0, 0, 0, 0
e = do |yy|
loop do
while x > xmin
yy << [x, y]
x -= 1
xmin -= 1
while y < ymax


Many types of functions produce data amenable for charting in the xy dimensions. These functions are defined as f(x[n]) = y. To transform this function into an f(x) = y function, provide a projection that takes a set of (x[n], y) values and picks a single (x,y) pair. In practice the set (x[n], y) values must be limited to fit within memory/runtime bounds... ergo for a given x there must be a limited number of x[n-1].

In summary a projection can produce xy data from x[n]y data provided:

  • there is one value y for each (x[n])
  • the number of x[n-1] is limited for a given x

Many types of data end up satisfying these conditions, in particular measurement data. Measurement data can be modeled as ty. If measurement data can be overridden then add a dimension to track when the data was recieved, ie tty.

thinkerbot /
Last active August 29, 2015 14:14
Clean merge

One way to merge nested configs cleanly:

  • flatten key paths
  • merge

This approach overcomes the issue of performing a deep merge when an overlay has the ability to specify a value or a hash. This does not deal with merging array values.

Here is the issue:

c1 = {"a" => 1}

RSpec issues with include

We're extracting modules from intellisource and that's a good engineering practice. We can localize and document the footprint of modules by requiring and including them where they're used.

module HelperModule

class HelperClass

thinkerbot / pipe-exec
Last active August 29, 2015 14:06
Hold open a fifo from both ends, allowing input from external processes
#!/usr/bin/env ruby
require 'optparse'
options = {
:direction => "in"
} do |opts|
opts.banner = %{
usage: pipe-exec [options] FIFO_FILE COMMAND....
thinkerbot / xyz.cql
Created August 26, 2014 16:48
Examples of cql queries
CREATE KEYSPACE example WITH REPLICATION = { 'class' : 'SimpleStrategy', 'replication_factor' : 3 };
USE example;
create table xyz (
xpart int,
x int,
z int,
y int,


Starting/stopping processes occurs in serial, ergo make sure the start/stop commands go quickly or the whole thing hangs. I It's a good idea to use scripts to setup the execution environment so you have something fully reproducible. Here is the command to get the ruby environment via rvm.

rvm info environment | sed -n -e 's/: \{1,\}/=/' -e 's|.rvm/bin:.*|.rvm/bin:$PATH"|' -e 's/    \([A-Z]\)/export \1/p'

An example usage:

check process process_name with pidfile /path/to/

start program = "/usr/bin/env -i KEY=VALUE /path/to/ /path/to/monit_exec start /path/to/ /path/to/stdin_file /path/to/stdout_file /path/to/logfile.log commmand args..."