Skip to content

Instantly share code, notes, and snippets.

@sam95
Last active June 13, 2019 19:45
Show Gist options
  • Save sam95/d7aed31770883bd272728ad0483629d4 to your computer and use it in GitHub Desktop.
Save sam95/d7aed31770883bd272728ad0483629d4 to your computer and use it in GitHub Desktop.
Installation and Getting started with Apache-Kafka for OSX

Apache Kafka is a highly-scalable publish-subscribe messaging system that can serve as the data backbone in distributed applications. With Kafka’s Producer-Consumer model it becomes easy to implement multiple data consumers that do live monitoring as well persistent data storage for later analysis. You can assume that RabbitMQ is similar to Kafka, You do get an option for message-sending or Broadcasting. The installation process for OSX is as below.

  1. The best way to install the latest version of the Kafka server on OS X and to keep it up to date is via Homebrew.

     brew search kafka
     brew install kafka 
    
  2. The above commands will install a dependency called zookeeper which is required to run kafka. Start the zookeeper service.

     zkserver start
    
  3. While Zookeeper is running, Start the kafka server which handles both producer and consumer parts. Locate the bin folder below cd /usr/local/Cellar/kafka/<kafka-version-number>/bin. In my example, the version which I am running is 0.11.0.0. This will keep kafka running.

     cd /usr/local/Cellar/kafka/0.11.0.0/bin
     sh kafka-server-start /usr/local/etc/kafka/server.properties
    
  4. In a new terminal, Start the consumer which will recieve events or messages.

     cd /usr/local/Cellar/kafka/0.11.0.0/bin
     sh kafka-console-consumer --zookeeper localhost:2181 --topic sample_topic --from-beginning
    
  5. Note that, the topic determines to where the producer is producing the event, and where the consumer is listening to. In simple words, Keep the topic same in both producer and consumer. Here the topic is sample_topic

  6. In a new terminal, Start the producer which will produce the events.

     cd /usr/local/Cellar/kafka/0.11.0.0/bin
     sh kafka-console-producer --broker-list localhost:9092 --topic sample_topic
    
  7. In the producer's terminal, You will get an angle prompt >. Type your messages and hit enter

  8. Receive your message in the consumer terminal.

@JunPyoL22
Copy link

JunPyoL22 commented Dec 14, 2018

I think this is working if your kafka server is running on "localhost:9092"
check the exact your "host:port" at the "/usr/local/etc/kafka/server.properties"
sh kafka-console-consumer --bootstrap-server localhost:9092 --topic sample_topic --from-beginning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment