Skip to content

Instantly share code, notes, and snippets.

@garystafford

garystafford/run_spark_batch.sh

Created December 18, 2022 03:40
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save garystafford/58d664f95bc14cbc21f522187de4dc70 to your computer and use it in GitHub Desktop.
Save garystafford/58d664f95bc14cbc21f522187de4dc70 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
run_spark_batch.sh
# set environment variables used by jobs
export BOOTSTRAP_SERVERS="kafka:29092"
export TOPIC_PURCHASES="demo.purchases"
cd /home/apache_spark_examples/
# run batch processing job
spark-submit spark_batch_kafka.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment