Kafka TLS config

sasl_plain.md

Variables:

• KAFKA_HOSTNAME - Hostname with your kafka service;
• USERNAME - username;
• PASSWORD - password

Kafka

listeners=PLAINTEXT://$KAFKA_HOSTNAME:9092,SASL_PLAIN://$KAFKA_HOSTNAME:9094
security.inter.broker.protocol=SASL_PLAIN
sasl.enabled.mechanisms=PLAIN

Broker/Zookeeper

Properties:

authProvider.sasl=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

JAAS config

Client {
   org.apache.zookeeper.server.auth.DigestLoginModule required
   username="$USERNAME"
   password="$PASSWORD";
};

Client

security.protocol=SASL_PLAINTEXT #no TLS
security.protocol=SASL_SL #Plaintext password over TLS
sasl.mechanism=PLAIN
sasl.username=$USERNAME
sasl.password=$PASSWORD

tls.md

References:

• https://github.com/confluentinc/librdkafka/wiki/Using-SSL-with-librdkafka

Variables:

• KAFKA_HOSTNAME - Hostname with your kafka service

Required prep-work

• Generate a Certificate Authority (CA) key and certificate.
• Create a key and certificate for the Kafka broker.
• Generate keys and certificates for each client that will connect to Kafka.

Configure Kafka Broker with Zookeeper:

Place the broker's key and certificate in the Kafka broker's keystore

Configure the Kafka broker to use SSL by setting the following config:

listeners=PLAINTEXT://$KAFKA_HOSTNAME:9092,SSL://$KAFKA_HOSTNAME:9093
ssl.keystore.location=/path/to/keystore.jks
ssl.keystore.password=your_keystore_password
ssl.key.password=your_key_password
ssl.truststore.location=/path/to/truststore.jks
ssl.truststore.password=your_truststore_password

Configure Zookeeper:

Enable SSL configuration and specify the path to Zookeeper's truststore and keystore via properties file:

secureClientPort=2181
ssl.keyStore.location=/path/to/zookeeper.keystore.jks
ssl.keyStore.password=your_keystore_password
ssl.trustStore.location=/path/to/zookeeper.truststore.jks
ssl.trustStore.password=your_truststore_password

# To require authentication of clients use "require", else "none" or "request"
ssl.client.auth = required

Client configuration

Assuming you're using librdkafka you need to place key and certificates onto file system

Configure client as follows:

security.protocol=ssl
ssl.key.location=/path/to/client_keystore.key
ssl.certificate.location=/path/to/client_certificate.pem
# CA certificate file for verifying the broker's certificate.
ssl.ca.location=/path/to/ca_certificate.pem