Created
June 13, 2018 10:44
-
-
Save skonto/db2192ab81b3aa36f9c5f67db627eb1c to your computer and use it in GitHub Desktop.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Dispatcher passes the following command to mesos: | |
18/06/13 10:30:45 INFO MesosClusterScheduler: Command for the driver (test):./bin/spark-submit --name test --master mesos://zk://master.mesos:2181/mesos --driver-cores 1.0 --driver-memory 1000M --class org.apache.spark.examples.SparkPi --conf spark.executor.extraJavaOptions="-Dexecutor.test.param1=\"value 1\" -Dexecutor.test.param2=value\\ 2 -Dexecutor.test.path=\$PATH" --conf spark.mesos.containerizer=mesos --conf spark.driver.supervise=false --conf spark.app.name=test --conf spark.driver.memory=1000M --conf spark.mesos.executor.docker.image=skonto/spark-test:test --conf spark.driver.cores=1.0 --conf spark.driver.extraJavaOptions="-Dspark.test.param1=\"value 1\" -Dspark.test.param2=value\\ 2 -Dspark.test.path=\$PATH" $MESOS_SANDBOX/spark-examples_2.11-2.3.0.jar 10000 | |
spark-class will run this at the executor side: | |
exec /usr/lib/jvm/jre1.8.0_152/bin/java -cp '/opt/spark/dist/conf/:/opt/spark/dist/jars/*:/etc/hadoop/' '-Dexecutor.test.param1=value 1' '-Dexecutor.test.param2=value 2' '-Dexecutor.test.path=$PATH' -Xmx1024m org.apache.spark.executor.CoarseGrainedExecutorBackend --driver-url spark://CoarseGrainedScheduler@ip-10-0-0-77.us-west-1.compute.internal:33887 --executor-id 9 --hostname 10.0.0.77 --cores 3 --app-id bddf96ce-45f6-45f7-ac83-5a3622cafc41-0032-driver-20180613103044-0001 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment