Skip to content

Instantly share code, notes, and snippets.

@crakjie

crakjie/Kafka-lag.md

Last active August 21, 2018 14:29
Show Gist options
  • Save crakjie/dcabfac4951b36957c118e9e02b89449 to your computer and use it in GitHub Desktop.
Save crakjie/dcabfac4951b36957c118e9e02b89449 to your computer and use it in GitHub Desktop.
Download ZIP
Kafka lag
Raw
Kafka-lag.md

Lag sum by consumer.

./kafka-consumer-groups.sh --new-consumer  --bootstrap-server localhost:9092 --describe --group csa_aggregator | tail -n +3 | awk '{ sum[$7] += $3 } END { for(i in sum) print i, sum[i] }'

Speed by consumer , the 10 number is '10 second' of sampling. the result is in message by secondes.

paste <(./kafka-consumer-groups.sh --new-consumer  --bootstrap-server localhost:9092 --describe --group csa_aggregator | tail -n +3 | awk '{ sum[$7] += $3 } END { for(i in sum) print i, sum[i] }' ) \
<(sleep 10 && ./kafka-consumer-groups.sh --new-consumer  --bootstrap-server localhost:9092 --describe --group csa_aggregator | tail -n +3 | awk '{ sum[$7] += $3 } END { for(i in sum) print i, sum[i] }') | \
awk '{ sum[$1] += ($4 - $2) /10   } END { for(i in sum) print i, sum[i] }'
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment