Skip to content

Instantly share code, notes, and snippets.

@carlosroman
Last active November 21, 2023 10:15
Show Gist options
  • Save carlosroman/e4e92dbad36c533f8bfbbfec2d5f5dbf to your computer and use it in GitHub Desktop.
Save carlosroman/e4e92dbad36c533f8bfbbfec2d5f5dbf to your computer and use it in GitHub Desktop.
Run Kafka locally (with Docker) and Datadog Agent config to pull metrics from it
---
version: "3.9"
services:
zookeeper:
image: quay.io/debezium/zookeeper:2.3
ports:
- 2181:2181
- 2888:2888
- 3888:3888
kafka:
image: quay.io/debezium/kafka:2.3
labels:
com.datadoghq.ad.checks: '{"kafka":{"init_config":{"is_jmx":true},"instances":[{"jmx_url":"service:jmx:rmi://localhost:9999/jndi/rmi://0.0.0.0:9999/jmxrmi","port":9999}]}}'
ports:
- 9092:9092
- 9999:9999
links:
- zookeeper
environment:
- ZOOKEEPER_CONNECT=zookeeper:2181
- JMXPORT=9999
- JMX_PORT=9999
- JMXHOST=0.0.0.0
---
version: "3.9"
services:
zookeeper:
image: quay.io/debezium/zookeeper:2.3
ports:
- 2181:2181
- 2888:2888
- 3888:3888
kafka:
image: quay.io/debezium/kafka:2.3
ports:
- 9092:9092
- 9999:9999
links:
- zookeeper
environment:
- ZOOKEEPER_CONNECT=zookeeper:2181
- JMXPORT=9999
- JMX_PORT=9999
- JMXHOST=0.0.0.0
---
## All options defined here are available to all instances.
#
init_config:
## @param is_jmx - boolean - required
## Whether or not this file is a configuration for a JMX integration.
#
is_jmx: true
## @param collect_default_metrics - boolean - optional - default: false
## Whether or not the check should collect all default metrics.
#
collect_default_metrics: true
## @param new_gc_metrics - boolean - optional - default: false
## Set to true to use better metric names for garbage collection metrics.
## jvm.gc.cms.count => jvm.gc.minor_collection_count
## jvm.gc.major_collection_count
## jvm.gc.parnew.time => jvm.gc.minor_collection_time
## jvm.gc.major_collection_time
## The default value is false to ensure backward compatibility.
#
new_gc_metrics: true
instances:
## @param host - string - required
## Kafka host to connect to.
#
# - host: 0.0.0.0
- jmx_url: service:jmx:rmi://localhost:9999/jndi/rmi://0.0.0.0:9999/jmxrmi
port: 9999
@carlosroman
Copy link
Author

kafka_conf.yaml should be in /etc/datadog-agent/conf.d/kafka.d/conf.yaml

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment