Kafka command-line tools (examples in OSX)

kafka-command.md

Kafka command line tools

List (not complete)

• zookeeper-server-start.sh - starting Zookeeper
• kafka-server-start.sh - start Kafka
• kafka-topics.sh - manage topics in Kafka
• kafka-console-producer.sh - script for sending messages to Kafka topic
• kafka-console-consumer.sh - script for consuming messages from Kafka topic
• kafka-run-class.sh - script for running different tools (list of tools can be found here)

Example use

The following paths are standard paths on OSX when Kafka is installed via Brew. At the time of writing current Kafka version is 0.11.0.1. All the commands are executed from within /usr/local/Cellar/kafka/0.11.0.1/libexec/bin

Ramp up Zookeeper:

./zookeeper-server-start.sh ../config/zookeeper.properties

Ramp up Kafka:

./kafka-server-start.sh ../config/server.properties

Create a topic

./kafka-topics.sh --create  --topic example-topic --zookeeper localhost:2181 --partitions 1 --replication-factor 1
./kafka-topics.sh --list

Create a consumer

./kafka-console-consumer.sh --topic example-topic --from-beginning --bootstrap-server localhost:9092

Produce message

./kafka-console-producer.sh --topic example-topic --broker-list localhost:9092
> {"event":"EXAMPLE", "data":{"id":"123", "name":"Piotr"}, "metadata":{"version":"001"}, "timestamp":"2017-11-05T00:00:00Z" }

Check offsets

./kafka-run-class.sh --topic example-topic --zookeeper zookeeper-server.com:2181 --group example-group

For create a topic,
--zookeeper is deprecated
use --bootstrap-server
https://kafka.apache.org/quickstart

By the way, these commands are all installed for me here (could be newer):

brew install kafka
ls -lah /opt/homebrew/opt/kafka/bin