Skip to content

Instantly share code, notes, and snippets.

@vpipkt

vpipkt/make-distro.sh

Created August 26, 2016 21:30
Show Gist options
  • Save vpipkt/b00d9696867910e65c2132d365438414 to your computer and use it in GitHub Desktop.
Save vpipkt/b00d9696867910e65c2132d365438414 to your computer and use it in GitHub Desktop.
Download ZIP
build tar.gz binaries for distribution of Spark 1.5.0 for Scala 2.11 ... for yarn
Raw
make-distro.sh
#!/usr/bin/env
git clone git@github.com:apache/spark
pushd spark
git checkout v1.5.0
./dev/change-scala-version 2.11
./make-distribution.sh --name scala_2.11-without-hadoop --tgz -Phadoop-provided -Dscala-2.11 -Pyarn -DskipTests
@vpipkt
Copy link
Author

vpipkt commented Aug 26, 2016

don't forget to export SPARK_DIST_CLASSPATH=$(hadoop classpath) in the conf/spark-env.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment