Skip to content

Instantly share code, notes, and snippets.

Tristan Reid tristanreid

Block or report user

Report or block tristanreid

Hide content and notifications from this user.

Learn more about blocking users

Contact Support about this user’s behavior.

Learn more about reporting abuse

Report abuse
View GitHub Profile
@tristanreid
tristanreid / ExecutionAppExamples.scala
Last active Oct 2, 2015
Examples of how (and how not) to call Scalding's ExecutionApp
View ExecutionAppExamples.scala
import com.twitter.scalding.typed.TypedPipe
import com.twitter.scalding._
import scala.util.{Failure, Success}
// If you want to understand what's going on, read the code of Execution and ExecutionApp (it's not long)
// Here's the highlights: The main of ExecutionApp - executes job for you, as long as you return Execution[Unit]
// Execution: Consider flatMap, zip, unit
// unit: since you have to return Execution[Unit] at some point, this is handy. Nice with .zip, for example
// zip: combine Executions to execute in parallel for fun and profit
@tristanreid
tristanreid / KMeans2Caller.scala
Last active Oct 2, 2015
Scalding ExecutionApp for KMeans example
View KMeans2Caller.scala
/*
To execute https://github.com/twitter/scalding/blob/master/scalding-core/src/main/scala/com/twitter/scalding/examples/KMeans.scala
and wrap results in a file.
Execute locally like this:
scala -classpath target/project-0.0.1-jar-with-dependencies.jar com.mycompany.project.KMeans2Caller \
--local \
--clusters <num clusters> \
--input ../work/kmeansData.tsv \
--output kout.tsv \
You can’t perform that action at this time.