Skip to content

Instantly share code, notes, and snippets.

@merlin-quix
Last active January 23, 2023 07:20
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save merlin-quix/f2b9b4f4d93be7ea8928c9b489b9e5cc to your computer and use it in GitHub Desktop.
Save merlin-quix/f2b9b4f4d93be7ea8928c9b489b9e5cc to your computer and use it in GitHub Desktop.
Step 1 in the procedure "Creating a Kafka Consumer" from this article: https://www.quix.io/blog/send-timeseries-data-to-kafka-python/
from kafka import KafkaConsumer
import json
import pandas as pd
# Consume all the messages from the topic but do not mark them as 'read' (enable_auto_commit=False)
# so that we can re-read them as often as we like.
consumer = KafkaConsumer('transactions',
group_id='test-consumer-group',
bootstrap_servers=['localhost:9092'],
value_deserializer=lambda m: json.loads(m.decode('utf-8')),
auto_offset_reset='earliest',
enable_auto_commit=False)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment