Skip to content

Instantly share code, notes, and snippets.

@ayubmalik
Last active March 5, 2024 17:24
Show Gist options
  • Star 13 You must be signed in to star a gist
  • Fork 4 You must be signed in to fork a gist
  • Save ayubmalik/9300bbd8f62ce401c4d1f69eae0cb5d7 to your computer and use it in GitHub Desktop.
Save ayubmalik/9300bbd8f62ce401c4d1f69eae0cb5d7 to your computer and use it in GitHub Desktop.
Kafka REST proxy with Docker compose. Everything required to get Confluent REST proxy docker images working so you can post messages to consumers with curl etc
---
version: '3.5'
networks:
default:
name: kafka-net
services:
zookeeper:
image: confluentinc/cp-zookeeper:4.1.1
hostname: zookeeper
ports:
- "32181:32181"
environment:
ZOOKEEPER_CLIENT_PORT: 32181
ZOOKEEPER_TICK_TIME: 2000
ZOOKEEPER_SYNC_LIMIT: 2
kafka:
image: confluentinc/cp-kafka:4.1.1
hostname: kafka
ports:
- "29092:29092"
depends_on:
- zookeeper
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:32181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
schema-registry:
image: confluentinc/cp-schema-registry:4.1.1
hostname: schema-registry
ports:
- "38081:38081"
depends_on:
- kafka
environment:
SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: zookeeper:32181
SCHEMA_REGISTRY_HOST_NAME: schema-registry
SCHEMA_REGISTRY_LISTENERS: http://schema-registry:38081
SCHEMA_REGISTRY_DEBUG: "true"
kafka-rest:
image: confluentinc/cp-kafka-rest:4.1.1
hostname: kafka-rest
ports:
- "38082:38082"
depends_on:
- schema-registry
environment:
KAFKA_REST_ZOOKEEPER_CONNECT: zookeeper:32181
KAFKA_REST_SCHEMA_REGISTRY_URL: schema-registry:38081
KAFKA_REST_HOST_NAME: kafka-rest
KAFKA_REST_LISTENERS: http://kafka-rest:38082
@ayubmalik
Copy link
Author

ayubmalik commented Jun 4, 2018

Kafka REST Proxy

Run this file with docker-compose i.e. docker-compose -f docker-compose.yaml

Get a list of topics

curl http://localhost:38082/topics

Full docs here: https://docs.confluent.io/current/kafka-rest/docs/intro.html#quickstart

@apristen
Copy link

apristen commented Oct 25, 2021

Can't start with my config:

    kafka-rest-1:
    image: confluentinc/cp-kafka-rest:6.2.1
    hostname: kafka-rest-1
    container_name: kafka-rest-1
    depends_on:
      - zookeeper-1
      - broker-1
    ports:
      - "8082:8082"
    environment:
      KAFKA_REST_HOST_NAME: kafka-rest-1
      KAFKA_REST_LISTENERS: http://0.0.0.0:8082
      KAFKA_REST_ZOOKEEPER_CONNECT: 192.168.100.14:2181
      KAFKA_REST_BOOTSTRAP_SERVERS: 192.168.100:14:9092
    networks:
      - mynet

in logs it displays:

root@lab:/home/pvadmin/Kafka-Docker# docker-compose up kafka-rest-1
zookeeper-1 is up-to-date
broker-1 is up-to-date
Recreating kafka-rest-1 ... done
Attaching to kafka-rest-1
kafka-rest-1    | ===> User
kafka-rest-1    | uid=1000(appuser) gid=1000(appuser) groups=1000(appuser)
kafka-rest-1    | ===> Configuring ...
kafka-rest-1    | ===> Running preflight checks ...
kafka-rest-1    | ===> Check if Zookeeper is healthy ...
kafka-rest-1    | SLF4J: Class path contains multiple SLF4J bindings.
kafka-rest-1    | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
kafka-rest-1    | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
kafka-rest-1    | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
kafka-rest-1    | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
kafka-rest-1    | log4j:WARN No appenders could be found for logger (io.confluent.admin.utils.cli.ZookeeperReadyCommand).
kafka-rest-1    | log4j:WARN Please initialize the log4j system properly.
kafka-rest-1    | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
kafka-rest-1    | ===> Check if Kafka is healthy ...
kafka-rest-1    | SLF4J: Class path contains multiple SLF4J bindings.
kafka-rest-1    | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
kafka-rest-1    | SLF4J: Found binding in [jar:file:/usr/share/java/cp-base-new/slf4j-simple-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
kafka-rest-1    | SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
kafka-rest-1    | SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
kafka-rest-1    | log4j:WARN No appenders could be found for logger (io.confluent.admin.utils.cli.KafkaReadyCommand).
kafka-rest-1    | log4j:WARN Please initialize the log4j system properly.
kafka-rest-1    | log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
kafka-rest-1 exited with code 1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment