How to run KubernetesSuite in Spark

gistfile1.txt

1. Install minikube
2. Start minikube with enough cpus and memory
   minikube start --memory='8196mb' --cpus=4
3. The Pod of spark doesn't specify systemaccount, so it is "default". Spark will create pod. So we should give enough
   permission to "default" systemaccount. Create role by kubectl and bind the role to systemaccount default
   kubectl create role default --verb=get,list,watch,create,update,patch,delete --resource=pods,pods/status
   kubectl create rolebinding default-binding --role=default --serviceaccount=default:default --namespace=default
4. Build Spark images. Remember to build PySpark image too.
   ./bin/docker-image-tool.sh -m -t dev -p resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/python/Dockerfile build
5. Run test.
   build/sbt -Dhadoop.version=2.7.0 -Phive -Phive-thriftserver -Pkubernetes -Pkubernetes-integration-tests "kubernetes-integration-tests/test-only *.KubernetesSuite"
   
Useful kubectl commands:
  kubectl get pods
  kubectl logs <pod>
  kubectl describe pod <pod>

`minikube dashboard` can show kubernetes cluster status in browser.

# This is another way from https://github.com/apache/spark/pull/30735
minikube delete
minikube start --cpus 12 --memory 16384
kubectl create namespace spark-integration-test
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: ServiceAccount
metadata:
  name: spark
  namespace: spark-integration-test
EOF
kubectl create clusterrolebinding spark-role --clusterrole=edit --serviceaccount=spark-integration-test:spark --namespace=spark-integration-test
dev/make-distribution.sh --pip --tgz -Pkubernetes
resource-managers/kubernetes/integration-tests/dev/dev-run-integration-tests.sh --spark-tgz `pwd`/spark-3.2.0-SNAPSHOT-bin-3.2.0.tgz  --service-account spark --namespace spark-integration-test