Skip to content

Instantly share code, notes, and snippets.

@toddleo
toddleo / sparkJob.scala
Last active August 17, 2018 06:03
A slow Kafka + Spark Structured Streaming example with CSV
import scalaz._, Scalaz._
import reflect.runtime.universe._
import org.apache.spark.sql._
import org.apache.spark.sql.functions._
import org.apache.spark.sql.types._
object MyJob {
def main(args: Array[String]): Unit = {
implicit val spark = SparkSession.builder.appName("Simple Application").getOrCreate()
@toddleo
toddleo / term-doc-matrix.R
Created May 16, 2017 15:06
Generate term-document matrix via **tm** package, and convert to dataframe.
library(tm)
library(magrittr)
docs <- c( "The Indians were taking on the Rays on Monday night, alright? The Indians won, 8-7, and Lonnie Chisenhall and Francisco Lindor both hit home runs, alright? Brad Miller got three hits while continuing to use a pink bat, alright?"
,"That's now three alrights, in three consecutive sentences. What does that get us? It gets us movie star/fanny pack enthusiast Matthew McConaughey wearing an alright hat. He was taking in the game while filming a movie in the area this month.")
# To view the corpus, uncomment the following line:
# docs %>% VectorSource %>% VCorpus %>% inspect
tdm <- docs %>% VectorSource %>% SimpleCorpus %>% TermDocumentMatrix
@toddleo
toddleo / .cvimrc
Last active August 29, 2015 14:24
" Settings
set autofocus
set cncpcompletion
set smoothscroll
set hud
set autohidecursor
set typelinkhints
let scrollduration = 500
let searchlimit = 40
@toddleo
toddleo / christmas_tree.r
Last active December 8, 2017 09:48
A fractal Christmas tree plot in R
# http://www.r-bloggers.com/merry-christmas-5/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+RBloggers+%28R+bloggers%29
# Each row is a 2x2 linear transformation
# Christmas tree
L <- matrix(
c(0.03, 0, 0 , 0.1,
0.85, 0.00, 0.00, 0.85,
0.8, 0.00, 0.00, 0.8,
0.2, -0.08, 0.15, 0.22,
-0.2, 0.08, 0.15, 0.22,
0.25, -0.1, 0.12, 0.25,