Kafka Notes

kafka.md

Kafka Docs

• Kafka Docs
  • Adding and removing topics
  • Modifying topics
  • Increasing replication factor
• Replication tools
• Balancing Kafka on JBOD

Create Kafka Topic

./bin/kafka-topics.sh \
  --create \
  --zookeeper localhost:2181 \
  --topic my_topic \
  --replication-factor 1 \
  --partitions 2

Adding Partitions to a Topic

./bin/kafka-topics.sh \
  --zookeeper zk_host:port/chroot \
  --alter \
  --topic my_topic \
  --partitions 40 \

• Note: While Kafka allows us to add more partitions, it is NOT possible to decrease number of partitions of a Topic. In order to achieve this, you need to delete and re-create your Topic.

Deleting a topic

./bin/kafka-topics.sh \
  --zookeeper zk_host:port/chroot \
  --delete \
  --topic my_topic_name

• Note: Topic deletion option is disabled by default. To enable it set the server config:

delete.topic.enable=true

Describe Kafka Topic

./bin/kafka-topics.sh \
  --describe --zookeeper localhost:2181 \
  --topic my_topic

Print out the current assignment in JSON format.

./bin/kafka-reassign-partitions.sh \
    --zookeeper zk1.xyu.io:2181,zk2.xyu.io:2181/kafka \
    --broker-list '1,2,3' \
    --topics-to-move-json-file topic.json \
    --generate

Increasing replication factor

./bin/kafka-reassign-partitions.sh \
  --zookeeper localhost:2181 \
  --reassignment-json-file  increase-replication-factor.json \
  --execute

cat increase-replication-factor.json
  {"version":1,
  "partitions":[{"topic":"foo","partition":0,"replicas":[5,6,7]}]}