Skip to content

Instantly share code, notes, and snippets.

@akiatoji

akiatoji/kafka_connect_zookeeper.md

Created October 27, 2014 15:55
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save akiatoji/f1a5ca319325788cf287 to your computer and use it in GitHub Desktop.
Save akiatoji/f1a5ca319325788cf287 to your computer and use it in GitHub Desktop.
Download ZIP
Why can't Kafka connect to zookeeper?
Raw
kafka_connect_zookeeper.md

Kafka -> Zookeeper

Kafka seems to try IPv6 address to connect to Zookeeper first, but most hadoop installation does not use IPv6 (well, CDH5 anyways).

So you might get a half-working Kafka installation like the following:

[cdh5@cdh5 kafka_2.10-0.8.1.1]$ bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
Created topic "test".

[cdh5@cdh5 kafka_2.10-0.8.1.1]$ bin/kafka-topics.sh --list --zookeeper localhost:2181
[2014-10-27 08:40:52,460] WARN Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.net.ConnectException: Connection refused
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
	at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1146)
test

Kafka is working when you try to list the topics above, but it's trying IPv6 (because localhost resolves to both IPv6 and IPv4 addresses) first and spewing out the connection refused error.

If you get this, stop IPv6 resolution, or have Zookeeper accept connection on IPv6 interfaces.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment