Skip to content

Instantly share code, notes, and snippets.

@bsikander

bsikander/spark-default.conf

Last active March 23, 2017 09:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save bsikander/54264349b49e6227c6912eb14d344b8c to your computer and use it in GitHub Desktop.
Save bsikander/54264349b49e6227c6912eb14d344b8c to your computer and use it in GitHub Desktop.
Download ZIP
Default configuration file for SLAVE
Raw
spark-default.conf
spark.streaming.backpressure.enabled true
spark.eventLog.enabled true
spark.worker.cleanup.enabled=true
# check for outdated stuff every 30 minutes
spark.worker.cleanup.interval=1800
# remove stuff older than 7 days
spark.worker.cleanup.appDataTtl=604800
#### configurable ####
spark.driver.memory 2g
spark.executor.memory 2g
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment