Skip to content

Instantly share code, notes, and snippets.

@skonto

skonto/gist:7b658cf1130e8c73048c6f6b5e77857c

Last active July 31, 2019 12:52
Show Gist options
  • Save skonto/7b658cf1130e8c73048c6f6b5e77857c to your computer and use it in GitHub Desktop.
Save skonto/7b658cf1130e8c73048c6f6b5e77857c to your computer and use it in GitHub Desktop.
Download ZIP
restart
Raw
gistfile1.txt
I0731 12:45:00.713186 10 controller.go:206] SparkApplication spark/spark-pi was updated, enqueueing it
I0731 12:45:00.713227 10 controller.go:246] Starting processing key: "spark/spark-pi"
I0731 12:45:00.713404 10 event.go:221] Event(v1.ObjectReference{Kind:"SparkApplication", Namespace:"spark", Name:"spark-pi", UID:"ec2313f6-b390-11e9-a01c-0e17384ad88e", APIVersion:"sparkoperator.k8s.io/v1beta1", ResourceVersion:"25340148", FieldPath:""}): type: 'Normal' reason: 'SparkApplicationSpecUpdateProcessed' Successfully processed spec update for SparkApplication spark-pi
I0731 12:45:00.713735 10 controller.go:206] SparkApplication spark/spark-pi was updated, enqueueing it
I0731 12:45:00.718758 10 controller.go:569] Trying to update SparkApplication spark/spark-pi, from: [{spark-7ef7cf6385634dffb50fbf7dc0faf0b1 a0f15b10-b66e-44f4-bd66-e91f2f10562e 2019-07-31 12:44:26 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {RUNNING } map[spark-pi-1564577064335-exec-1:RUNNING spark-pi-1564577064335-exec-2:RUNNING] 1 1}] to [{spark-7ef7cf6385634dffb50fbf7dc0faf0b1 a0f15b10-b66e-44f4-bd66-e91f2f10562e 2019-07-31 12:44:26 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {RUNNING } map[spark-pi-1564577064335-exec-1:RUNNING spark-pi-1564577064335-exec-2:RUNNING] 1 1}]
I0731 12:45:00.718968 10 controller.go:253] Ending processing key: "spark/spark-pi"
I0731 12:45:00.719011 10 controller.go:246] Starting processing key: "spark/spark-pi"
I0731 12:45:00.719048 10 controller.go:748] Deleting pod spark-pi-driver in namespace spark
I0731 12:45:00.726044 10 spark_pod_eventhandler.go:58] Pod spark-pi-driver updated in namespace spark.
I0731 12:45:00.726075 10 spark_pod_eventhandler.go:95] Enqueuing SparkApplication spark/spark-pi for app update processing.
I0731 12:45:00.729886 10 controller.go:757] Deleting Spark UI Service spark-pi-ui-svc in namespace spark
I0731 12:45:00.730550 10 spark_pod_eventhandler.go:77] Pod spark-pi-driver deleted in namespace spark.
I0731 12:45:00.730579 10 spark_pod_eventhandler.go:95] Enqueuing SparkApplication spark/spark-pi for app update processing.
I0731 12:45:00.758285 10 controller.go:569] Trying to update SparkApplication spark/spark-pi, from: [{spark-7ef7cf6385634dffb50fbf7dc0faf0b1 a0f15b10-b66e-44f4-bd66-e91f2f10562e 2019-07-31 12:44:26 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {INVALIDATING } map[spark-pi-1564577064335-exec-1:RUNNING spark-pi-1564577064335-exec-2:RUNNING] 1 1}] to [{ a0f15b10-b66e-44f4-bd66-e91f2f10562e 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {PENDING_RERUN } map[] 0 0}]
I0731 12:45:00.763174 10 spark_pod_eventhandler.go:58] Pod spark-pi-1564577064335-exec-2 updated in namespace spark.
I0731 12:45:00.763201 10 spark_pod_eventhandler.go:95] Enqueuing SparkApplication spark/spark-pi for app update processing.
I0731 12:45:00.764565 10 spark_pod_eventhandler.go:58] Pod spark-pi-1564577064335-exec-1 updated in namespace spark.
I0731 12:45:00.764590 10 spark_pod_eventhandler.go:95] Enqueuing SparkApplication spark/spark-pi for app update processing.
I0731 12:45:00.769790 10 sparkapp_metrics.go:144] Exporting metrics for spark-pi; old status: {spark-7ef7cf6385634dffb50fbf7dc0faf0b1 a0f15b10-b66e-44f4-bd66-e91f2f10562e 2019-07-31 12:44:26 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {INVALIDATING } map[spark-pi-1564577064335-exec-1:RUNNING spark-pi-1564577064335-exec-2:RUNNING] 1 1} new status: { a0f15b10-b66e-44f4-bd66-e91f2f10562e 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {PENDING_RERUN } map[] 0 0}
I0731 12:45:00.769866 10 controller.go:253] Ending processing key: "spark/spark-pi"
I0731 12:45:00.769958 10 controller.go:246] Starting processing key: "spark/spark-pi"
I0731 12:45:00.770034 10 controller.go:748] Deleting pod spark-pi-driver in namespace spark
I0731 12:45:00.770367 10 controller.go:206] SparkApplication spark/spark-pi was updated, enqueueing it
I0731 12:45:00.773741 10 controller.go:757] Deleting Spark UI Service spark-pi-ui-svc in namespace spark
I0731 12:45:00.777916 10 controller.go:569] Trying to update SparkApplication spark/spark-pi, from: [{spark-7ef7cf6385634dffb50fbf7dc0faf0b1 a0f15b10-b66e-44f4-bd66-e91f2f10562e 2019-07-31 12:44:26 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {INVALIDATING } map[spark-pi-1564577064335-exec-1:RUNNING spark-pi-1564577064335-exec-2:RUNNING] 1 1}] to [{ a0f15b10-b66e-44f4-bd66-e91f2f10562e 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {PENDING_RERUN } map[] 0 0}]
I0731 12:45:00.790111 10 sparkapp_metrics.go:144] Exporting metrics for spark-pi; old status: {spark-7ef7cf6385634dffb50fbf7dc0faf0b1 a0f15b10-b66e-44f4-bd66-e91f2f10562e 2019-07-31 12:44:26 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {INVALIDATING } map[spark-pi-1564577064335-exec-1:RUNNING spark-pi-1564577064335-exec-2:RUNNING] 1 1} new status: { a0f15b10-b66e-44f4-bd66-e91f2f10562e 0001-01-01 00:00:00 +0000 UTC 0001-01-01 00:00:00 +0000 UTC {spark-pi-ui-svc 30485 10.0.2.27:30485 spark-pi-driver} {PENDING_RERUN } map[] 0 0}
I0731 12:45:00.790267 10 controller.go:253] Ending processing key: "spark/spark-pi"
I0731 12:45:00.790314 10 controller.go:246] Starting processing key: "spark/spark-pi"
I0731 12:45:00.790339 10 controller.go:555] SparkApplication spark/spark-pi pending rerun
I0731 12:45:00.798420 10 controller.go:557] Resources for SparkApplication spark/spark-pi successfully deleted
I0731 12:45:00.798553 10 submission.go:63] spark-submit arguments: [/opt/spark/bin/spark-submit --class org.apache.spark.examples.SparkPi --master k8s://https://172.30.0.1:443 --deploy-mode cluster --conf spark.kubernetes.namespace=spark --conf spark.app.name=spark-pi --conf spark.kubernetes.driver.pod.name=spark-pi-driver --conf spark.kubernetes.container.image=lightbend/spark:2.1.1-OpenShift-2.4.3-rh-2.12 --conf spark.kubernetes.container.image.pullPolicy=Always --conf spark.kubernetes.submission.waitAppCompletion=false --conf spark.kubernetes.driver.label.sparkoperator.k8s.io/app-name=spark-pi --conf spark.kubernetes.driver.label.sparkoperator.k8s.io/launched-by-spark-operator=true --conf spark.kubernetes.driver.label.sparkoperator.k8s.io/submission-id=5754794d-9537-4a85-97d7-2d4d10ef23ea --conf spark.driver.memory=500m --conf spark.kubernetes.authenticate.driver.serviceAccountName=spark-sa --conf spark.kubernetes.driver.label.version=2.4.3 --conf spark.kubernetes.executor.label.sparkoperator.k8s.io/app-name=spark-pi --conf spark.kubernetes.executor.label.sparkoperator.k8s.io/launched-by-spark-operator=true --conf spark.kubernetes.executor.label.sparkoperator.k8s.io/submission-id=5754794d-9537-4a85-97d7-2d4d10ef23ea --conf spark.executor.instances=4 --conf spark.executor.memory=500m --conf spark.kubernetes.executor.label.version=2.4.3 local:///opt/spark/examples/jars/spark-examples_2.12-2.4.3.jar 100000000]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment