Skip to content

Instantly share code, notes, and snippets.

@iht

iht/read_kafka.scala

Created January 27, 2020 21:51
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save iht/86132ae6457d69a9bec937c9bb1bb41d to your computer and use it in GitHub Desktop.
Save iht/86132ae6457d69a9bec937c9bb1bb41d to your computer and use it in GitHub Desktop.
Download ZIP
Read all unacknowledged messages in a Kafka topic with KafkaIO in Apache Beam
Raw
read_kafka.scala
KafkaIO.read[String, String]
.withBootstrapServers(config.broker)
.withTopic(config.kafkaTopic)
.withKeyDeserializer(classOf[StringDeserializer])
.withValueDeserializer(classOf[StringDeserializer])
.withStartReadTime(Instant.ofEpochMilli(1))
.withoutMetadata
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment