Skip to content

Instantly share code, notes, and snippets.

@bobbruno
Created January 12, 2018 10:30
Show Gist options
  • Save bobbruno/318a28666f3e86b6285bf4e74a15fa2c to your computer and use it in GitHub Desktop.
Save bobbruno/318a28666f3e86b6285bf4e74a15fa2c to your computer and use it in GitHub Desktop.
Load Stanford CoreNLP in Databricks Spark
val version = "3.7.0" // CoreNLP version the model will be used with
val model = s"stanford-corenlp-$version-models" // append "-english" to use the full English model
if (!sc.listJars.exists(jar => jar.contains(model))) {
import scala.sys.process._
s"wget http://repo1.maven.org/maven2/edu/stanford/nlp/stanford-corenlp/$version/$model.jar -O /tmp/$model.jar".!!
sc.addJar(s"/tmp/$model.jar")
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment