Tomas Tauber tomtau
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import com.twitter.scalding.mathematics.Matrix | |
| class BenchOldJob(args : Args) extends Job(args) { | |
| import Matrix._ | |
| val a0 = Tsv("/home/ttauber/scalding-bench/a0.tsv", ('x, 'y, 'v) ) | |
| .read. | |
| toMatrix[Int,Int,Double]('x,'y,'v) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import cascading.pipe.joiner._ | |
| import com.twitter.scalding.mathematics.Matrix2 | |
| import com.twitter.scalding.mathematics.FiniteHint | |
| import com.twitter.algebird.Group | |
| class BenchNewJob(args : Args) extends Job(args) { | |
| import Matrix2._ | |
| import com.twitter.scalding.mathematics.MatrixLiteral |
tomtau
/ Graph propagation (common subtrees)
Created
September 5, 2013 15:00
Generated job using the old Matrix API
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import com.twitter.scalding.mathematics.Matrix | |
| class BenchOldJob(args : Args) extends Job(args) { | |
| import Matrix._ | |
| val a0 = Tsv("/home/ttauber/scalding-bench/a0.tsv", ('x, 'y, 'v) ) | |
| .read. | |
| toMatrix[Int,Int,Double]('x,'y,'v) | |
| (a0 * a0 * a0 * a0 * a0 * a0 * a0 * a0).write( Tsv("/home/ttauber/scalding-bench/aold.tsv")) |
tomtau
/ Graph propagation (common subtrees)
Created
September 5, 2013 15:02
Generated job using the new Matrix API
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import cascading.pipe.joiner._ | |
| import com.twitter.scalding.mathematics.Matrix2 | |
| import com.twitter.scalding.mathematics.FiniteHint | |
| import com.twitter.algebird.Group | |
| class BenchNewJob(args : Args) extends Job(args) { | |
| import Matrix2._ | |
| import com.twitter.scalding.mathematics.MatrixLiteral |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import com.twitter.scalding.mathematics.Matrix | |
| class BenchOldJob(args : Args) extends Job(args) { | |
| import Matrix._ | |
| val a0 = Tsv("/home/ttauber/scalding-bench/a0.tsv", ('x, 'y, 'v) ) | |
| .read. | |
| toMatrix[Int,Int,Double]('x,'y,'v) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import cascading.pipe.joiner._ | |
| import com.twitter.scalding.mathematics.Matrix2 | |
| import com.twitter.scalding.mathematics.FiniteHint | |
| import com.twitter.algebird.Group | |
| class BenchNewJob(args : Args) extends Job(args) { | |
| import Matrix2._ | |
| import com.twitter.scalding.mathematics.MatrixLiteral |
tomtau
/ TSV Matrix and Job Generator
Created
September 5, 2013 15:11
Quick and dirty script for generating jobs in the old and new Matrix API as well as corresponding TSV
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| package tsvgen | |
| import java.io._ | |
| import scala.util.Random | |
| object TsvGenerator { | |
| def generateTsv(matrixParam: (String, Long, Long)): Unit = { | |
| val (filename, rows, cols) = matrixParam | |
| val writer = new PrintWriter(new File(filename)) |
tomtau
/ Optimization or Untyped Pipe overhead
Created
September 5, 2013 15:18
Generated job using the old Matrix API
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import com.twitter.scalding.mathematics.Matrix | |
| class BenchOldJob(args : Args) extends Job(args) { | |
| import Matrix._ | |
| val a0 = Tsv("/home/ttauber/scalding-bench/a0.tsv", ('x, 'y, 'v) ) | |
| .read. | |
| toMatrix[Int,Int,Double]('x,'y,'v) |
tomtau
/ Optimization or Untyped Pipe overhead
Created
September 5, 2013 15:19
Generated job using the new Matrix API
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import cascading.pipe.joiner._ | |
| import com.twitter.scalding.mathematics.Matrix2 | |
| import com.twitter.scalding.mathematics.FiniteHint | |
| import com.twitter.algebird.Group | |
| class BenchNewJob(args : Args) extends Job(args) { | |
| import Matrix2._ | |
| import com.twitter.scalding.mathematics.MatrixLiteral |
tomtau
/ Matrix chain multiplication
Created
September 5, 2013 15:21
Generated job using the old Matrix API
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| import com.twitter.scalding._ | |
| import com.twitter.scalding.mathematics.Matrix | |
| class BenchOldJob(args : Args) extends Job(args) { | |
| import Matrix._ | |
| val a0 = Tsv("/home/ttauber/scalding-bench/a0.tsv", ('x, 'y, 'v) ) | |
| .read. | |
| toMatrix[Int,Int,Double]('x,'y,'v) |
OlderNewer