Skip to content

Instantly share code, notes, and snippets.

@aviemzur

aviemzur/spark_local_cluster_mode

Last active March 24, 2017 13:00
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aviemzur/0c33af90c9e21b826f2c0311d97fa7da to your computer and use it in GitHub Desktop.
Save aviemzur/0c33af90c9e21b826f2c0311d97fa7da to your computer and use it in GitHub Desktop.
Download ZIP
Spark Local Cluster Mode
Raw
spark_local_cluster_mode
local-cluster[2, 2, 128]
Raw
vm_options
-Dspark.testing=true
-Dspark.test.home=/path/to/spark/installation
-Dspark.testing.reservedMemory=31457280
-Dspark.driver.memory=128m
-Dspark.executor.memory=128m
-Dspark.ui.enabled=true
-Dspark.ui.port=8088
-Dspark.streaming.backpressure.enabled=true
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment