Skip to content

Instantly share code, notes, and snippets.

@kavimaluskam
Last active January 30, 2019 09:59
Show Gist options
  • Save kavimaluskam/55cdb3b96439fc1ec8d517a1f5ce2c73 to your computer and use it in GitHub Desktop.
Save kavimaluskam/55cdb3b96439fc1ec8d517a1f5ce2c73 to your computer and use it in GitHub Desktop.
Cheat sheet for Standalone Kafka Connect with JSON deserialisation
# Enter Kafka Connect Shell Client
make connect-shell
# Kafka Connect with connect-standalone setting, connecting via postgres
connect-standalone dev/connect-standalone.properties dev/source-postgres-incremental.properties
# Expected console-consumer output from topic: app-users
# {
# "schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":false,"name":"users"},
# "payload":{"id":101,"first_name":"Cass","last_name":"Truss","email":"ctruss0@salon.com"}
# }
# {
# "schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":false,"name":"users"},
# "payload":{"id":102,"first_name":"Joete","last_name":"Scowcroft","email":"jscowcroft1@gizmodo.com"}
# }
# {
# "schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":false,"name":"users"},
# "payload":{"id":103,"first_name":"Celine","last_name":"Wortley","email":"cwortley2@bizjournals.com"}
# }
# {
# "schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":false,"name":"users"},
# "payload":{"id":104,"first_name":"Othilia","last_name":"Gumme","email":"ogumme3@issuu.com"}
# }
# {
# "schema":{"type":"struct","fields":[{"type":"int32","optional":false,"field":"id"},{"type":"string","optional":false,"field":"first_name"},{"type":"string","optional":false,"field":"last_name"},{"type":"string","optional":false,"field":"email"}],"optional":false,"name":"users"},
# "payload":{"id":105,"first_name":"Maynord","last_name":"Buttrick","email":"mbuttrick4@mashable.com"}
# }
# ...
# Exit Docker when Kafka Connect is done
(Ctrl-C)
exit
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment