Skip to content

Instantly share code, notes, and snippets.

@timvw
Last active February 18, 2019 13:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save timvw/72a03beffd14189ad8df5e41bb311757 to your computer and use it in GitHub Desktop.
Save timvw/72a03beffd14189ad8df5e41bb311757 to your computer and use it in GitHub Desktop.
generate --dependencies and --repositories arguments for spark from (effective) pom file
import java.io._
import scala.xml.XML
import ammonite.ops._
import ammonite.ops.ImplicitWd._
val tempFile = File.createTempFile("tmp", "tmp")
tempFile.deleteOnExit()
val f = tempFile.getAbsolutePath
val res = %%('mvn, "help:effective-pom", s"-Doutput=${f}")
val xml = XML.loadFile(f)
val deps = {
(xml \\ "dependencies" \\ "dependency")
.filter(x => (x \\ "scope").text == "compile")
.map(x => ((x \\ "groupId").text, (x \\ "artifactId").text, (x \\ "version").text))
.map(x => s"${x._1}:${x._2}:${x._3}")
.toList.distinct
.mkString(",")
}
println(s"--dependencies ${deps}")
val repos = {
(xml \\ "repositories" \\ "repository")
.map(x => (x \\ "url").text)
.toList.distinct
.mkString(",")
}
println(s"--repositories $repos")
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment