Skip to content

Instantly share code, notes, and snippets.

@ag-ramachandran
Created August 5, 2022 06:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save ag-ramachandran/285b9e84f0188d6d095feed354719c7f to your computer and use it in GitHub Desktop.
Save ag-ramachandran/285b9e84f0188d6d095feed354719c7f to your computer and use it in GitHub Desktop.
Kafka connect without schema registry
---
version: '3'
services:
zookeeper:
image: confluentinc/cp-zookeeper:6.2.5
hostname: zookeeper
container_name: zookeeper
ports:
- "2181:2181"
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
kafka:
image: confluentinc/cp-kafka:6.2.5
hostname: kafka
container_name: kafka
depends_on:
- zookeeper
ports:
- "9092:9092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
connect:
build:
context: .
dockerfile: Dockerfile-KafkaConnect
hostname: connect
container_name: connect
depends_on:
- zookeeper
- kafka
ports:
- 8083:8083
- 8888:8888
environment:
CONNECT_BOOTSTRAP_SERVERS: "kafka:9092"
CONNECT_REST_PORT: 8083
KAFKA_DEBUG: "true"
JAVA_DEBUG_PORT: "*:8888"
CONNECT_REST_ADVERTISED_HOST_NAME: "connect"
CONNECT_GROUP_ID: compose-connect-group
CONNECT_CONFIG_STORAGE_TOPIC: connect-configs
CONNECT_OFFSET_STORAGE_TOPIC: connect-offsets
CONNECT_STATUS_STORAGE_TOPIC: connect-status
CONNECT_KEY_CONVERTER_SCHEMAS_ENABLE: "false"
CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE: "false"
CONNECT_INTERNAL_KEY_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_INTERNAL_VALUE_CONVERTER: "org.apache.kafka.connect.json.JsonConverter"
CONNECT_KEY_CONVERTER: "org.apache.kafka.connect.converters.ByteArrayConverter"
CONNECT_VALUE_CONVERTER: "org.apache.kafka.connect.converters.ByteArrayConverter"
CONNECT_LOG4J_ROOT_LOGLEVEL: "INFO"
CONNECT_LOG4J_LOGGERS: "org.apache.kafka.connect.runtime.rest=WARN,org.reflections=ERROR"
CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_STATUS_STORAGE_REPLICATION_FACTOR: "1"
CONNECT_PLUGIN_PATH: '/usr/share/java'
CLASSPATH: '/usr/share/java/proto-serde-with-dependencies.jar'
@ConfieBlake
Copy link

Hi @ag-ramachandran , what are
context: .
dockerfile: Dockerfile-KafkaConnect

Seems the Dockerfile-KafkaConnect is not provided

@ag-ramachandran
Copy link
Author

Hello @ConfieBlake , Will provide a sample project for the whole scenario. Give us some-time

@justmike1
Copy link

Looking forward as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment