Useful Kafka Metrics

kafka-metrics.md

Kafka metrics:

• kafka.consumer.fetch.manager.records.consumed.rate
• kafka.consumer.fetch.manager.records.lag

Spring Kafka Listener metrics

• spring.kafka.listener.count
• spring.kafka.listener.max

Metrics queries:

• sum(align(1m, ts(kafka.consumer.fetch.manager.records.consumed.rate, space=${env} and service=${service} and dc=${dc})), service, dc)
• sum(align(1m, ts(kafka.consumer.fetch.manager.records.lag, space=${env} and service=${service} and instance=${instance} and dc=${dc} )), service, dc, instance, topic)
• sum(align(1m, ts(spring.kafka.listener.count, space=${env} and service=${service} and dc=${dc})), service, dc, result)
• avg(align(1m, ts(spring.kafka.listener.max, space=${env} and service=${service} and dc=${dc})), service, dc, result)

Spring configuration:

management:
  tracing:
    sampling:
      probability: 1.0

  metrics:
    tags:
      org: ${vcap.application.organization_name}
      space: ${vcap.application.space_name}
      application: ${vcap.application.space_name}
      service: ${vcap.application.name}
      instance: ${vcap.application.name}-${CF_INSTANCE_INDEX:0}