Skip to content

Instantly share code, notes, and snippets.

View andypetrella's full-sized avatar

Andy Petrella andypetrella

288 followers · 73 following
View GitHub Profile
@andypetrella
andypetrella / QPlot.snb
Last active March 9, 2016 02:33
Super basic qplot implementation on Scala collection, using R backend as renderer... in 74 lines :-)
{
"metadata" : {
"name" : "QPlot",
"user_save_timestamp" : "1970-01-01T01:00:00.000Z",
"auto_save_timestamp" : "1970-01-01T01:00:00.000Z",
"language_info" : {
"name" : "scala",
"file_extension" : "scala",
"codemirror_mode" : "text/x-scala"
},
@andypetrella
andypetrella / GraphFramesExample.snb
Last active March 4, 2016 18:25 — forked from deanwampler/GraphFramesExample.snb
Databrick's Python example for the new GraphFrame API ported to Scala and Spark Notebook
{
"metadata" : {
"name" : "GraphFramesExample",
"user_save_timestamp" : "1970-01-01T01:00:00.000Z",
"auto_save_timestamp" : "1970-01-01T01:00:00.000Z",
"language_info" : {
"name" : "scala",
"file_extension" : "scala",
"codemirror_mode" : "text/x-scala"
},
@andypetrella
andypetrella / build.sbt
Last active January 3, 2016 12:09
An sbt configuration for https://github.com/NightHacking/LambdasHacking. Put this in "Code" and launch sbt. IT REQUIRES SBT > 0.13.1
libraryDependencies += "junit" % "junit" % "4.11" % "test"
libraryDependencies += "com.novocode" % "junit-interface" % "0.10" % "test"
javaSource in Test := baseDirectory.value / "test"
testOptions += Tests.Argument(TestFrameworks.JUnit, "-q", "-v")
javacOptions ++= Seq("-source", "1.8")
@andypetrella
andypetrella / README.md
Last active December 22, 2015 10:58
Scrapping the Belgium PICC

How to scrap the Belgium PICC

Download ESRI PICC data

Use the Query operation on the service, aksing for:

  • f=json json format
  • fields=*
  • geometryType=esriGeometryEnvelope
  • geometry=...
@andypetrella
andypetrella / MRTweet.scala
Last active December 22, 2015 06:19
Expressiveness, conciseness and behavior
val l:List[A]
val m[K,V]:A=>List[(K, V)]
val r[V]:List[V]=>V
val s[K]:K=>K
l .flatMap(m)
.groupBy(_._1) // map phase
.mapValues(_.map(_._2))
.groupBy{case (x, xs) => s(x) } // shuffled
.mapValues(_.map{case (x,xs) => (x, r(xs))}) // reduced
@andypetrella
andypetrella / mr.scala
Created August 29, 2013 15:38
MR scala
//def
val l:List[A]
val m[K,V]:A=>List[(K, V)]
val r[V]:List[V]=>V
val s[K]:K=>K
//mr
l.flatMap(m)
.groupBy(_._1)
.mapValues(_.map(_._2))
@andypetrella
andypetrella / SparkAkka.scala
Last active December 20, 2015 22:39
Spark-bd blog entries
package spark
import spark.util.AkkaUtils
object SparkAkka {
private[this] lazy val config = be.bigdata.p2.conf.root.getConfig("deploy.akka")
lazy val name = config.getString("name")
lazy val host = config.getString("host")
lazy val port = config.getInt("port")
lazy val actorSystem = AkkaUtils.createActorSystem(name, host, port)._1
@andypetrella
andypetrella / conway.scala
Created August 7, 2013 06:10
Game of Life (Conway) in scala
object conway {
def neighbours(p:(Int, Int)) =
for {
dx <- List(-1, 0, 1)
dy <- if (dx == 0) List(-1, 1) else List(-1, 0, 1)
} yield (dx+p._1, dy+p._2)
def step(cells:List[(Int, Int)]) =
cells
@andypetrella
andypetrella / Co.scala
Last active December 19, 2015 17:59
Co-Contra variance in Scala
package cocontra
import model._
object Co extends App {
val kids = List(new Kid(9))
val humans:List[Human] = kids
println(humans)
@andypetrella
andypetrella / mat.scala
Last active December 17, 2015 22:49
Matrix Algebra and type level programming ~> compile time checks?
/*
First implementation of a Matrix String Interpolator.
This is a manipulable matrix strucutre
mat"""
| 1 | 2
| 3 |
"""
*/
object Mat {