Skip to content

Instantly share code, notes, and snippets.

@thattommyhall
thattommyhall / emr.sh
Last active December 15, 2015 10:39
Fixing UTF-8 issues in redshift imports with ruby 1.9.3 and elastic mapreduce. The reducer is pretty dumb, tried to get it working with the identity reducer but it didnt work. You might want to add some more cleanup in either the mapper or the reducer to trim text fields that are too big, make sure number fields are exported correctly, NaN vs nu…
./elastic-mapreduce --create --stream \
--input s3n://YOUR_BUCKET/PATH_TO_FILES/ \
--mapper s3n://YOUR_BUCKET/utf8-cleanup.rb \
--reducer s3n://YOUR_BUCKET/utf8-cleanup-reducer.rb \
--output s3n://YOUR_BUCKET/PATH_TO_FILES_CLEANED \
--bootstrap-action "s3n://YOUR_BUCKET/ruby193.sh" \
--debug \
--num-instances 20
@thattommyhall
thattommyhall / santa.clj
Created February 15, 2013 15:12
A fairly faithful translation into clojure of Simon Peyton Jones's solution to The Santa Claus Problem using haskells STM from the Beautiful Code book. Had to use try/catch as clojures STM does not have retry or orElse, not sure how idomatic it is. Full project at https://github.com/thattommyhall/santa-claus
(ns santa.core)
(declare join-group)
(defn start-thread
[fn]
(.start
(Thread. fn)))
(defn random-delay []
@thattommyhall
thattommyhall / q.clj
Last active December 12, 2015 03:28
Hacker cup qualifier
(def letters
(set "abcdefghijklmnopqrstuvwxyz"))
;; If you don't know clojure it might be interesting that sets are functions of their elements
;; and return either nil or the element, so can be used as predicates
(defn counts [s]
(frequencies (filter letters (str/lower-case s))))
;; Here we filter the string for letters we care about (after lowercasing it)
(def fibs
(map first (iterate (fn [[a b]] [b (+ a b)]) [1 1])))
(defn fibs-from [n]
(drop-while #(< % n) fibs))
(take 5 fibs)
; (1 1 2 3 5)
(take 5 (fibs-from 1000000000000))
@thattommyhall
thattommyhall / fib.coffee
Last active December 11, 2015 20:49
Coffeescript fib
fib = (n) ->
r = [1,1]
for i in [2...n]
r.push(r[i-1] + r[i-2])
r
console.log fib(5)
input {
tcp {
port => 5544
type => syslog
}
udp {
port => 5544
type => syslog
}
}
(def __
(fn [n]
(let [
big-p (fn [n]
(let [sn (str n)]
(str sn
(clojure.string/reverse sn))))
small-p (fn [n]
(let [sn (str n)
@thattommyhall
thattommyhall / gist:4254789
Created December 11, 2012 00:56
4clojure 101
;Levenshtein Distance - Hard
;Given two sequences x and y, calculate the <a href="https://secure.wikimedia.org/wikipedia/en/wiki/Levenshtein_distance">Levenshtein distance</a> of x and y, i. e. the minimum number of edits needed to transform x into y. The allowed edits are:<br/><br/>- insert a single item<br/>- delete a single item<br/>- replace a single item with another item<br/><br/>WARNING: Some of the test cases may timeout if you write an inefficient solution!
;tags - seqs
;restricted -
(ns offline-4clojure.p101
(:use clojure.test))
(def __
(fn [s t]
(let [m (count s)
; Graph Connectivity - Hard
; Given a graph, determine whether the graph is connected. A connected graph is such that a path exists between any two given nodes.<br/><br/>-Your function must return true if the graph is connected and false otherwise.<br/><br/>-You will be given a set of tuples representing the edges of a graph. Each member of a tuple being a vertex/node in the graph.<br/><br/>-Each edge is undirected (can be traversed either direction).
; tags - graph-theory
; restricted -
(ns offline-4clojure.p91
(:use clojure.test))
(def __
(fn [s]
; Analyze a Tic-Tac-Toe Board - Hard
; A <a href="http://en.wikipedia.org/wiki/Tic-tac-toe">tic-tac-toe</a>
; board is represented by a two dimensional vector.
; X is represented by :x, O is represented by :o, and empty is represented by :e.
; A player wins by placing three Xs or three Os in a horizontal, vertical, or diagonal row.
; Write a function which analyzes a tic-tac-toe board and returns :x if X has won, :o if O has won, and nil if neither player has won.
; tags - game
; restricted -
(ns offline-4clojure.p73
(:use clojure.test))