Kafka CLI Cheat Sheets

kafkacli.md

Create topic:

kafka-topics.sh --zookeeper 127.0.0.1:2181 --topic first_topic --create --partitions 3 --replication-factor 1

List topic:

kafka-topics.sh --zookeeper 127.0.0.1:2181 --list

Describe topic:

kafka-topics.sh --zookeeper 127.0.0.1:2181 --topic first_topic --describe

Delete topic:

kafka-topics.sh --zookeeper 127.0.0.1:2181 --topic second_topic --delete

Produce message

kafka-console-produce.sh --broker-list 127.0.0.1:9092 --topic first_topic

Produce message with parameters

kafka-console-produce.sh --broker-list 127.0.0.1:9092 --topic first_topic --producer-property acks=all

Produce message and create new topic if it not exists

kafka-console-produce.sh --broker-list 127.0.0.1:9092 --topic new_topic

Consumer new messages

kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --topic first_topic 

Consume messages from beginning

kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --topic first_topic --from-beginning

Consumer group

kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --topic first_topic --group my_first_application

Consumer group reading messages from beginning

kafka-console-consumer.sh --bootstrap-server 127.0.0.1:9092 --topic first_topic --group my_first_application 

List consumer groups

kafka-consumer-groups --bootstrap-server 127.0.0.1:9092 --topic --list

Describe consumer groups

kafka-consumer-groups --bootstrap-server localhost:9092 --descript --group my_first_application

Reseting offsets (check doc for period)

kafka-consumer-groups --bootstrap-server localhost:9092 --group my-first-application --reset-offsets --to-earliest --execute --topic first_topic

Reseting offsets --shift-by

kafka-consumer-groups --bootstrap-server localhost:9092 --group my-first-application --reset-offsets --shift-by 2 --execute --topic first_topic

Producer with keys

kafka-console-producer --broker-list 127.0.0.1:9092 --topic first_topic --property parse.key=true --property key.separator=,
> key,value
> another key,another value

Consumer with keys

kafka-console-consumer --bootstrap-server 127.0.0.1:9092 --topic first_topic --from-beginning --property print.key=true --property key.separator=,