Skip to content

Instantly share code, notes, and snippets.

View blancas's full-sized avatar

Armando Blancas blancas

  • Palo Alto, Calif.
View GitHub Profile
1 101 5.0
1 102 3.0
1 103 2.5
2 101 2.0
2 102 2.5
2 103 5.0
2 104 2.0
3 101 2.5
3 104 4.0
3 105 4.5
@blancas
blancas / gist:5586206
Created May 15, 2013 18:35
Log for the hang when sample runs in an SBT project
sbt run
[info] Set current project to Simple (in build file:/Users/ablancas/xtra/tools/spark-0.7.0/zap.simple/)
[info] Running Simple
13/05/15 11:34:26 INFO slf4j.Slf4jEventHandler: Slf4jEventHandler started
13/05/15 11:34:27 INFO storage.BlockManagerMaster: Registered BlockManagerMaster Actor
13/05/15 11:34:27 INFO storage.MemoryStore: MemoryStore started with capacity 333.7 MB.
13/05/15 11:34:27 INFO storage.DiskStore: Created local directory at /var/folders/l_/5fbf5w8s1xj53jm1nh936rp40000gn/T/spark-local-20130515113427-c366
13/05/15 11:34:27 INFO network.ConnectionManager: Bound socket to port 51900 with id = ConnectionManagerId(wifi-ablancas-mbp.dhcp.carrieriq.com,51900)
13/05/15 11:34:27 INFO storage.BlockManagerMaster: Trying to register BlockManager
13/05/15 11:34:27 INFO storage.BlockManagerMasterActor$BlockManagerInfo: Registering block manager wifi-ablancas-mbp.dhcp.carrieriq.com:51900 with 333.7 MB RAM
@blancas
blancas / gist:5586194
Created May 15, 2013 18:32
Sample that works at the spark shell
def manhattan(p: Map[Int, Double], q: Map[Int,Double]): Double = {
val diffs = for ((k, x) <- p; y = q.get(k); if y != None)
yield { Math.abs(x - y.get) }
diffs.sum
}
val input = sc.textFile("intro.csv")
val data = input.map(line => {
val Array(user, item, rating) = line.split(",")
@blancas
blancas / gist:5507033
Last active December 16, 2015 22:29
stateful map with a state monad
(use '[blancas.morph core monads])
(def cards [{:balance 30} {:balance 25}])
(def due 100)
(run-state (mapm #(monad [due get-state
app (state (min due (:balance %)))
_ (put-state (- due app))]
(state (assoc % :applied-balance app))) cards) due)
@blancas
blancas / writer-bit
Created January 21, 2013 23:23
A bit of the Writer monad.
(deftype Writer [value output]) ;; output must implement Monoid.
(extend-type Writer
Functor
(fun [this f]
(monad [x this] (return this (f x))))
Applicative
(<*> [this m]
(if (coll? m)
(monad [f this vs (seqm m)] (return this (apply f vs)))