Skip to content

Instantly share code, notes, and snippets.

@rchatsiri
Created October 25, 2019 10:11
Show Gist options
  • Save rchatsiri/8fc38bd68460f2b7776b09352dc805a8 to your computer and use it in GitHub Desktop.
Save rchatsiri/8fc38bd68460f2b7776b09352dc805a8 to your computer and use it in GitHub Desktop.
Example run spark-operator
pi.yaml:
apiVersion: "sparkoperator.k8s.io/v1beta1"
kind: SparkApplication
metadata:
name: spark-pi
namespace: spark
spec:
type: Scala
mode: cluster
image: "lightbend/spark:2.0.0-OpenShift-2.4.0"
imagePullPolicy: Always
mainClass: org.apache.spark.examples.SparkPi
mainApplicationFile: "local:///opt/spark/examples/jars/spark-examples_2.11-2.4.0.jar"
sparkVersion: "2.4.0"
restartPolicy:
type: Never
driver:
cores: 0.1
coreLimit: "200m"
memory: "512m"
labels:
version: 2.4.0
serviceAccount: spark-sa
executor:
cores: 1
instances: 1
memory: "512m"
labels:
version: 2.4.0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment