Skip to content

Instantly share code, notes, and snippets.

@confluentgist
Created July 6, 2020 17:22
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save confluentgist/31ef8041eb8c07064e7ad0acdc6ba83b to your computer and use it in GitHub Desktop.
Save confluentgist/31ef8041eb8c07064e7ad0acdc6ba83b to your computer and use it in GitHub Desktop.
A script to produce randomized user records into a Kafka topic
import sys
import time
import os
import json
import random
from dateutil import parser
from confluent_kafka import Producer
import names # https://pypi.org/project/names/
import random
TOPIC = 'users'
cities = ["Mountain View", "San Francisco", "Daly City", "San Mateo", "San Bruno", "Millbrae", "Burlingame", "Foster City", "Belmont", "San Carlos"]
is_customer=["true", "false"]
producer = Producer({
'bootstrap.servers': '<your-bootstrap-server>',
'broker.version.fallback': '0.10.0.0',
'api.version.fallback.ms': 0,
'sasl.mechanisms': 'PLAIN',
'security.protocol': 'SASL_SSL',
'sasl.username': '<your-kafka-api-key>',
'sasl.password': '<your-kafka-api-secret>'
})
for i in range (5000):
msg ='"name":"{}", "address":"{}", "age":{}, "is_customer":{}'.format(names.get_full_name(), cities[i%10], random.randint(20,40), is_customer[i%2])
# create a json object
msg = '{' + msg + '}'
assert json.loads(msg)
producer.produce(TOPIC, msg)
producer.flush()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment