Skip to content

Instantly share code, notes, and snippets.

@bsikander

bsikander/spark-env.sh

Created March 23, 2017 10:28
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bsikander/42f76d7a8e4079098d8a2df3cdee8ee0 to your computer and use it in GitHub Desktop.
Save bsikander/42f76d7a8e4079098d8a2df3cdee8ee0 to your computer and use it in GitHub Desktop.
Download ZIP
Spark-env.sh on MASTER
Raw
spark-env.sh
SPARK_MASTER_IP=<MASTER_IP>
SPARK_MASTER_PORT=7077
SPARK_DAEMON_JAVA_OPTS="-Dspark.deploy.recoveryMode=ZOOKEEPER -Dspark.deploy.zookeeper.url=<OUR_IPS> -Dspark.deploy.zookeeper.dir=/spark"
SPARK_WORKER_DIR=/<BASE>/spark/worker_dir
SPARK_LOCAL_DIRS=/<BASE>/spark/local_dirs
SPARK_CONF_DIR=/<BASE>/spark-master/config
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment