Skip to content

Instantly share code, notes, and snippets.

@ijan10

ijan10/pandas_udf_spark_submit.sh Secret

Last active Oct 20, 2019
Embed
What would you like to do?
spark-submit --conf spark.driver.maxResultSize=50g --executor-memory 70g --conf spark.executor.memoryOverhead=40g --conf spark.driver.memory=70g --executor-cores 14 --conf spark.yarn.am.memory=7g --conf spark.executor.extraJavaOptions=-XX:+UseG1GC --conf spark.yarn.am.cores=4 --master yarn --name create_ctr_model.py --verbose create_ctr_model.py --env=production --model_selected_feature_name=placement_type_name --running_date=2019_10_20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment