Skip to content

Instantly share code, notes, and snippets.

@gnzjgo
Last active December 9, 2022 14:26
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gnzjgo/f1a80186a301cd8770a946d02343bafd to your computer and use it in GitHub Desktop.
Save gnzjgo/f1a80186a301cd8770a946d02343bafd to your computer and use it in GitHub Desktop.
Sample of how to send dummy data to a Kinesis data stream
"""
Kinesis to Tinybird demo
Complement to this guide(tbd).
Create a virtual environment and install the dependencies.
```
python3 -mvenv .e
. .e/bin/activate
pip install boto3
```
Be sure your ~/.aws/credentials and ~/.aws/config files are OK according to [boto3 docs](https://github.com/boto/boto3#getting-started), that your account has the right privileges.
Start sending dummy data:
```
python3 send_to_kinesis.py
```
"""
import json
import boto3
from datetime import datetime
import time
import random
STREAM_NAME = "demo-tb-kinesis-stream"
products = ["sZzx0cUDX98","5d0cgAl5BTk","YY4YaHKh2jQ","p8Drpg_duLw","xFmXLq_KJxg","fSdBxY0NxVI","6cHumpSxTvs","Zu7A1GCSjZE","Fg15LdqpWrs"]
events = ["view","cart","sale"]
events_weights = [60, 33, 24]
while True:
event = {
'datetime': datetime.utcnow().isoformat(),
'event': random.choices(events, weights=events_weights,k=1)[0],
'product': random.choice(products)
}
print(event)
boto3.client('kinesis').put_record(
StreamName=STREAM_NAME,
Data=json.dumps(event),
PartitionKey="partitionkey")
time.sleep(0.1)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment