Skip to content

Instantly share code, notes, and snippets.

@dksifoua

dksifoua/Spark Multi node.sh

Last active October 14, 2019 15:14
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dksifoua/62bb4c9e83c3523a81d63943200cec1a to your computer and use it in GitHub Desktop.
Save dksifoua/62bb4c9e83c3523a81d63943200cec1a to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Spark Multi node.sh
# On the master
# The master will then launch et give you its address like: spark://IP:PORT
$ sudo ./spark-class org.apache.spark.deploy.master.Master
# On the workers
$ sudo ./spark-class org.apache.spark.deploy.worker.Worker spark://IP:PORT
# Start the shell
$ sudo ./spark-shell --master spark://IP:PORT
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment