uuhnaut69

I've been working with Apache Kafka for over 7 years. I inevitably find myself doing the same set of activities while I'm developing or working with someone else's system. Here's a set of Kafka productivity hacks for doing a few things way faster than you're probably doing them now. 🔥

• Show me all my Kafka topics and their partitions, replicas, and consumers
• Show me the contents of a topic
• Create a Kafka topic
• Produce messages to a Kafka topic
• Validate the schema of messages before producing to a topic
• Do all of this at a distance

Get the tools

uuhnaut69

version: "3.7"

services:
  mysql:
    container_name: mysql
    image: mysql:5.7
    command: ["mysqld", "--log-bin=mysql-bin", "--server-id=223344", "--binlog_format=row", "--expire_logs_days=1"]
    environment:
 MYSQL_ROOT_PASSWORD: root1234

uuhnaut69

---
version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.4.1
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"

uuhnaut69

postgres-cdc-kafka-connect-es-setup-guide

A tutorial setup ETL Postgres CDC -> Kafka Connect -> ES

Topology

                   +-------------+
                   |             |
                   |   Postgres  |
 | |