Skip to content

Instantly share code, notes, and snippets.

@geoHeil
Last active June 8, 2020 20:13
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save geoHeil/e80495f8eee1f5e119f3948a7c09c032 to your computer and use it in GitHub Desktop.
Save geoHeil/e80495f8eee1f5e119f3948a7c09c032 to your computer and use it in GitHub Desktop.
NiFi cannot connect to docker / fails to put record with timeout exception
version: "3"
services:
zookeeper: # the configuration manager
hostname: zookeeper
container_name: zookeeper
image: 'bitnami/zookeeper:3.6.1'
environment:
- ALLOW_ANONYMOUS_LOGIN=yes
nifi:
image: apache/nifi:1.11.4
ports:
- 8080:8080 # Unsecured HTTP Web Port
environment:
- NIFI_WEB_HTTP_PORT=8080
- NIFI_CLUSTER_IS_NODE=true
- NIFI_CLUSTER_NODE_PROTOCOL_PORT=8082
- NIFI_ZK_CONNECT_STRING=zookeeper:2181
- NIFI_ELECTION_MAX_WAIT=1 min
links:
- broker
broker:
image: confluentinc/cp-kafka:5.5.0
hostname: broker
container_name: broker
depends_on:
- zookeeper
ports:
- "29092:29092"
- "9092:9092"
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://broker:29092,PLAINTEXT_HOST://localhost:9092
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
@geoHeil
Copy link
Author

geoHeil commented Jun 8, 2020

When creating and listening to a topic as defined in:

docker-compose exec broker  kafka-topics --create --topic test --partitions 1 --replication-factor 1 --if-not-exists --zookeeper zookeeper:2181


docker-compose exec broker  \
kafka-topics --describe --topic test --zookeeper zookeeper:2181

docker-compose exec broker  \
  kafka-console-consumer --bootstrap-server localhost:29092 --topic test --from-beginning --max-messages 30

nifi does not publish any records for this dockerfile when trying to follow tutorials such as:

@geoHeil
Copy link
Author

geoHeil commented Jun 8, 2020

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment