Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
kafka installation on ubuntu 16.04
Install and configure Apache Kafka on Ubuntu 16.04
1. Install Java (JDK8)::
Add the repository
$ sudo add-apt-repository -y ppa:webupd8team/java
Update the metadata of the new repository and install JDK
$ sudo apt update && apt install oracle-java8-installer -y
Verify that JDK 8 is installed properly
$ java -version
You should see the output something like this
java version "1.8.0_111"
Java(TM) SE Runtime Environment (build 1.8.0_111-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.111-b14, mixed mode)
2. Install zookeeper::
$ sudo apt install zookeeperd
After the installation completes, ZooKeeper will be started as a daemon automatically. By default, it will listen on port 2181.
Confirm zookeeper is running on expected port::
$ netstat -ant | grep :2181
If everything's fine, you should see the following output:
tcp6 0 0 :::2181 :::* LISTEN
if after typing 'ruok' once connected to 'localhost', zookeeper will respond with 'imok' and close the session.
$ telnet localhost 2181
Trying ::1...
Connected to localhost.
Escape character is '^]'.
ruok <-- Type at empty prompt!
imokConnection closed by foreign host.
3. Download kafka from http://kafka.apache.org/downloads.html::
$ curl -O http://apache-mirror.rbc.ru/pub/apache/kafka/0.10.1.1/kafka_2.11-0.10.1.1.tgz
Untar and move binaries to /opt/kafka/:
$ sudo mkdir -p /opt/kafka && sudo tar -xvf kafka_2.11-0.10.1.1.tgz -C /opt/kafka/
4. Configure and run Kafka Server::
Turn on topic delete
$ sudo sed -i 's/#delete.topic.enable=true/delete.topic.enable=true/' /opt/kafka/kafka_2.11-0.10.1.1/config/server.properties
Start Kafka by running kafka-server-start.sh script
$ sudo /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-server-start.sh /opt/kafka/kafka_2.11-0.10.1.1/config/server.properties
Now, we can check listening ports:
- ZooKeeper : 2181
- Kafka
$ netstat -ant | grep -E ':2181|:9092'
If everything's fine, you should see the following output:
tcp6 0 0 :::9092 :::* LISTEN 1367/java
tcp6 0 0 :::2181 :::* LISTEN -
5. Test Server::
Open another session, and create a topic:
$ /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-topics.sh --create --topic topic-test --zookeeper localhost:2181 --partitions 1 --replication-factor 1
List available topics:
$ /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-topics.sh --list --zookeeper localhost:2181
Send message to topic as a producer via the 'kafka-console-producer.sh'::
echo "hello world" | /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic topic-test
*Consume* the send message::
$ /opt/kafka/kafka_2.11-0.10.1.1/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic topic-test --from-beginning
# referecing:
# https://gist.github.com/monkut/07cd1618102cbae8d587811654c92902
# https://devops.profitbricks.com/tutorials/install-and-configure-apache-kafka-on-ubuntu-1604-1/
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.