Skip to content

Instantly share code, notes, and snippets.

@rnbtechnology

rnbtechnology/read_from_kafka.py

Last active January 7, 2021 23:30
Show Gist options
  • Save rnbtechnology/7d79a5957229a2bf7a9f660214e4ff1b to your computer and use it in GitHub Desktop.
Save rnbtechnology/7d79a5957229a2bf7a9f660214e4ff1b to your computer and use it in GitHub Desktop.
Download ZIP
Raw
read_from_kafka.py
TOPIC_NAME = "topic_name"
KAFKA_BOOTSTRAP_SERVERS = "host1:port1,host2:port2"
# read data from Kafka
df = (
spark.readStream.format("kafka")
.option("kafka.bootstrap.servers", KAFKA_BOOTSTRAP_SERVERS)
.option("subscribe", TOPIC_NAME)
.load()
)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment