Skip to content

Instantly share code, notes, and snippets.

@will123195

will123195/IntelliJ Spark Scala Mongo.md

Last active March 6, 2018 15:27
Show Gist options
  • Save will123195/3ffcfb83ea2893633e9fb9a4ed8894e6 to your computer and use it in GitHub Desktop.
Save will123195/3ffcfb83ea2893633e9fb9a4ed8894e6 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
IntelliJ Spark Scala Mongo.md

New IntelliJ Project for Spark/Scala/Mongo

  1. Install IntelliJ
  2. IntelliJ > Preferences > Plugins > Install Scala > Restart IntelliJ
  3. Create New Project > Scala > SBT
    • JDK: 1.8
    • SBT: 0.13
    • Scala: 2.11
  4. Add this to build.sbt: 
    libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-core" % "2.2.0" % "provided",
  "org.apache.spark" %% "spark-sql" % "2.2.0" % "provided",
  "org.mongodb.spark" %% "mongo-spark-connector" % "2.2.0",
  "org.mongodb.scala" %% "mongo-scala-driver" % "2.1.0",
  "com.typesafe" % "config" % "1.3.1"
)
  5. Add this to project/plubins.sbt: 
    addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.5")
  6. Right click src/main/scala > New Package
  7. Right click package > New Scala Class
  8. Add New Configuration
    • Name: Run
    • Main Class: [your class]
    • Use classpath of module: [your module]
    • Before launch: Build
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment