Skip to content

Instantly share code, notes, and snippets.

@yjshen

yjshen/Sink streaming results continuously to Pulsar topics. Secret

Created July 15, 2019 07:12
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save yjshen/5e2658ad2be146d2c13f5a7d465fdd15 to your computer and use it in GitHub Desktop.
Save yjshen/5e2658ad2be146d2c13f5a7d465fdd15 to your computer and use it in GitHub Desktop.
Download ZIP
Raw
Sink streaming results continuously to Pulsar topics.
val ds = df
.selectExpr("__topic", "CAST(__key AS STRING)", "CAST(value AS STRING)") // the __topic field is used to choose the right topic for each record
.writeStream
.format("pulsar")
.option("service.url", "pulsar://localhost:6650")
.start()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment