DharmendraRathor/spark_submit_configuration.sh

Created October 4, 2019 19:09

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/DharmendraRathor/f5516555fb2f71cccaa3c32c89d4ad29.js"></script>
Save DharmendraRathor/f5516555fb2f71cccaa3c32c89d4ad29 to your computer and use it in GitHub Desktop.

Download ZIP

Spark submit partition and memory configuration

Raw

spark_submit_configuration.sh

	--conf spark.driver.memory=8G --num-executors=2 --executor-memory=14G --executor-cores=2

	Explanation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment