Skip to content

Instantly share code, notes, and snippets.

@iht

iht/read_kafka.scala

Last active January 27, 2020 22:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save iht/37c4a026d4665dab1f5e9a0545f88730 to your computer and use it in GitHub Desktop.
Save iht/37c4a026d4665dab1f5e9a0545f88730 to your computer and use it in GitHub Desktop.
Download ZIP
Read all unacknowledged messages in a Kafka topic with KafkaIO in Apache Beam
Raw
read_kafka.scala
KafkaIO.read[String, String]
.withBootstrapServers(config.broker)
.withTopic(config.kafkaTopic)
.withKeyDeserializer(classOf[StringDeserializer])
.withValueDeserializer(classOf[StringDeserializer])
.withStartReadTime(Instant.ofEpochMilli(1))
.withoutMetadata
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment