Skip to content

Instantly share code, notes, and snippets.

@jdye64
Last active September 16, 2020 13:28
Show Gist options
  • Save jdye64/e0caba690fb0b36be34487aa7697010d to your computer and use it in GitHub Desktop.
Save jdye64/e0caba690fb0b36be34487aa7697010d to your computer and use it in GitHub Desktop.
from custreamz import kafka
# How to connect to Kafka, brokers, partitions, security, etc ...
# Full list of configurations can be found at:
# https://github.com/edenhill/librdkafka/blob/master/CONFIGURATION.md
kafka_configs = {
"metadata.broker.list": "localhost:9092",
"group.id": "custreamz-client",
}
# Create a reusable Kafka Consumer client; "datasource"
consumer = kafka.Consumer(kafka_configs)
# Read 10,000 messages from `custreamz_tips` topic in CSV format.
tips_df = consumer.read_gdf(topic="custreamz_tips",
partition=0,
start=0,
end=10000,
message_format="CSV")
print(tips_df.head())
tips_df['tip_percentage'] = tips_df['tip'] / tips_df['total_bill'] * 100
# display average tip by dining party size
print(tips_df.groupby('size').tip_percentage.mean())
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment