Skip to content

Instantly share code, notes, and snippets.

@aneeshpanoli

aneeshpanoli/spak_conf.py

Last active January 24, 2020 18:02
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save aneeshpanoli/6c0a1185ad355ca3a0fae97254f172d4 to your computer and use it in GitHub Desktop.
Save aneeshpanoli/6c0a1185ad355ca3a0fae97254f172d4 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
spak_conf.py
def set_conf():
conf = SparkConf().setAppName("App")
conf = (conf.setMaster('local[*]')
.set('spark.executor.memory', '4G')
.set('spark.driver.memory', '16G')
.set('spark.driver.maxResultSize', '8G'))
return conf
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment