brianlow/kafka-from-cli.md

## kafka-from-cli.md

      
    Raw
  

              kafka-from-cli.md
            
          
    Assumed conditions:


Apache Kafka on Heroku cluster provisioned
Topic has been created named cli-test (but you can change the name used below)
Heroku CLI
Heroku CLI Kafka plugin
Homebrew, sed

Install C/C++ Kafka Library (librdkafka) and Kafka CLI tool (kafkacat)

For OS X, use the Homebrew command below. For others, see installation instructions at the links above.
brew install librdkafka kafkacat
Note: Requires librdkafka >= 0.9.1 (SSL support)
Export local environment variables

If you are running the consumer and producer at the same time from two command line windows, you'll need to export these variables on each command line. Replace my-app-with-kafka with your Heroku app name that has the Kafka add-on.
heroku config:get KAFKA_TRUSTED_CERT -a my-app-with-kafka > $PWD/ca.pem
heroku config:get KAFKA_CLIENT_CERT -a my-app-with-kafka > $PWD/cert.pem
heroku config:get KAFKA_CLIENT_CERT_KEY -a my-app-with-kafka > $PWD/cert.key
export KAFKA_URL="$(heroku config:get KAFKA_URL -a my-app-with-kafka | sed 's/kafka\+ssl\:\/\///g')"
export TOPIC=cli-test

Run the following two commands each in its own command line window and watch messages fly from the Consumer through the Kafka cluster to the Producer!

Consumer

This command will start listening and print to stdout any messages it receives from the Kafka cluster. Ctrl-C to stop.
kafkacat -C -t $TOPIC -b $KAFKA_URL \
-X security.protocol=ssl \
-X ssl.key.location=$PWD/cert.key \
-X ssl.ca.location=$PWD/ca.pem \
-X ssl.certificate.location=$PWD/cert.pem
Producer

This command will read input from stdin and send it to the Kafka cluster. Type a message followed by enter. Ctrl-D to stop.
kafkacat -P -t $TOPIC -b $KAFKA_URL \
-X security.protocol=ssl \
-X ssl.key.location=$PWD/cert.key \
-X ssl.ca.location=$PWD/ca.pem \
-X ssl.certificate.location=$PWD/cert.pem