clarksun/kafka_topic_msg_count.sh

Created October 9, 2017 06:36

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/clarksun/540336aff2b246177a1bc7b4aee977a2.js"></script>
Save clarksun/540336aff2b246177a1bc7b4aee977a2 to your computer and use it in GitHub Desktop.

Download ZIP

get kafka topic message count

Raw

kafka_topic_msg_count.sh

kafka-run-class kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic xxx --time -1 --offsets 1 | awk -F ':' '{sum += $3} END {print sum}'

enrique-filiage-movile commented Oct 11, 2019

Thanks!

Just a comment, parameter --offsets (also --max-wait-ms) is "DEPRECATED AND IGNORED"

My current CLI version is:

kafka-topics.sh --version
2.3.0 (Commit:fc1aaa116b661c8a)

Schachte commented Mar 25, 2020

How can you do this whilst still being able to pass in credentials?

swapnilbb commented May 6, 2020

ubuntu@prod-kafka-1:/opt/kafka/kafka_2.12-2.2.0$ bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:9092 --topic topic_1 --time -1 --offsets 1 | awk -F ":" '{sum += $3} END {print sum}'
6075

ubuntu@prod-kafka-1:/opt/kafka/kafka_2.12-2.2.0$ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topic_1 --property print.key=true --property key.separator="-" --from-beginning

^CProcessed a total of 0 messages
ubuntu@prod-kafka-1:/opt/kafka/kafka_2.12-2.2.0$

Apparently, this is not correct and it gives wrong information as seen above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment