Skip to content

Instantly share code, notes, and snippets.

View jakekdodd's full-sized avatar

Jake K. Dodd jakekdodd

  • San Francisco, CA
View GitHub Profile
@jakekdodd
jakekdodd / README.md
Created February 20, 2015 16:30
Example of kafka-python producer using Avro

Notes:

  • This uses the python loremipsum generator. You can replace g.generate_sentence() with your own string or generator of choice
  • Make sure to replace schema_path in the script with your own schema path
  • No guarantees that this is the best way to loop over records and publish them to Kafka, just a demo. Before adapting this script, take a look at the Avro documentation...the writer might not need to be instantiated for every record, for example. I'm just not sure.

Instructions

  1. Create a Kafka topic to test this on (default in the script is 'test')
  2. Replace schema_path with the path to user.avsc
  3. Open up a console Kafka consumer (see the 'quick start' section in Kafka's documentation)