This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ONM-THTRAN-01:CLDeepLearning longtran$ sbt compile | |
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=12G; support was removed in 8.0 | |
[info] Loading project definition from /Users/longtran/work/CLDeepLearning/project | |
[info] Set current project to CLDeep (in build file:/Users/longtran/work/CLDeepLearning/) | |
[error] Modules were resolved with conflicting cross-version suffixes in {file:/Users/longtran/work/CLDeepLearning/}cldeeplearning: | |
[error] org.apache.spark:spark-launcher _2.11, _2.10 | |
[error] org.apache.spark:spark-graphx _2.10, _2.11 | |
[error] org.json4s:json4s-ast _2.11, _2.10 | |
[error] org.apache.spark:spark-catalyst _2.10, _2.11 | |
[error] org.apache.spark:spark-network-shuffle _2.11, _2.10 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//https://github.com/deeplearning4j/nd4s/blob/master/build.sbt | |
name := "CLDeep" | |
scalaVersion := "2.10.6" | |
libraryDependencies ++= Seq( | |
"org.deeplearning4j" % "dl4j-spark_2.10" % "0.6.0" exclude("org.apache.spark","spark-core_2.10" ), | |
"org.apache.spark" % "spark-core_2.10" % "1.6.0" exclude("javax.servlet", "servlet-api") | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
//https://github.com/deeplearning4j/nd4s/blob/master/build.sbt | |
name := "CLDeep" | |
//scalaVersion := "2.10.6" | |
//crossScalaVersions := Seq("2.10.0", "2.10.6", "2.11.8") | |
//crossPaths := false | |
name := "CLDEEP" | |
version := "0.0.0" | |
libraryDependencies ++= Seq( |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
[error] Modules were resolved with conflicting cross-version suffixes in {file:/Users/longtran/work/CLDeepLearning/}cldeeplearning: | |
[error] org.apache.spark:spark-launcher _2.10, _2.11 | |
[error] org.apache.spark:spark-graphx _2.10, _2.11 | |
[error] org.json4s:json4s-ast _2.10, _2.11 | |
[error] org.apache.spark:spark-catalyst _2.10, _2.11 | |
[error] org.apache.spark:spark-network-shuffle _2.10, _2.11 | |
[error] com.typesafe.akka:akka-actor _2.10, _2.11 | |
[error] com.twitter:chill _2.10, _2.11 | |
[error] org.apache.spark:spark-streaming _2.10, _2.11 | |
[error] org.apache.spark:spark-sql _2.10, _2.11 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// our labelled sentences | |
case class AnnotatedText(isRhetorical: Boolean, content: String) | |
def toLabelledPoints(annotatedTextList: List[AnnotatedText]) ={ | |
//contributors: @rgcase, @ppremont | |
import org.apache.spark.mllib.classification.SVMWithSGD | |
import org.apache.spark.mllib.linalg.Vectors | |
import org.apache.spark.mllib.regression.LabeledPoint | |
import org.apache.spark.rdd.RDD | |
import org.apache.spark.{SparkConf, SparkContext} |