Skip to content

Instantly share code, notes, and snippets.

@Voronenko
Created July 3, 2023 18:56
Show Gist options
  • Save Voronenko/1d7b5acd741f6b0f6f411512e5c21a7e to your computer and use it in GitHub Desktop.
Save Voronenko/1d7b5acd741f6b0f6f411512e5c21a7e to your computer and use it in GitHub Desktop.
Quick local spark instance
version: '3'
services:
spark-master:
image: bde2020/spark-master:3.3.0-hadoop3.3
container_name: spark-master
ports:
- "8080:8080"
- "7077:7077"
environment:
- INIT_DAEMON_STEP=setup_spark
spark-worker-1:
image: bde2020/spark-worker:3.3.0-hadoop3.3
container_name: spark-worker-1
depends_on:
- spark-master
ports:
- "8081:8081"
environment:
- "SPARK_MASTER=spark://spark-master:7077"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment