Skip to content

Instantly share code, notes, and snippets.

@yangl

yangl/kafka-log-dump.md

Last active March 10, 2023 07:31
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save yangl/5260e97fc0ef6b753a35396892182ccc to your computer and use it in GitHub Desktop.
Save yangl/5260e97fc0ef6b753a35396892182ccc to your computer and use it in GitHub Desktop.
Download ZIP
kafka 日志 batch 消息数据情况分析。背景：容灾kafka集群磁盘暂用比上游生产集群大好多，查看上下游压缩格式、消息格式、batch占比
Raw
kafka-log-dump.md

kafka 日志 batch 消息数据情况分析

sh ~/kafka_2.13-3.4.0/bin/kafka-dump-log.sh  --deep-iteration --files ./00000000002687193593.log > dr-dump2.log

cat dr-dump2.log |grep 'count: '|awk -F 'count: ' '{print $2}'|awk '$1>1{print $1}'|wc -l


cat dr-dump2.log |grep 'count: '|awk -F 'count: ' '{print $2}'|awk '{print $1}'|wc -l
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment