Skip to content

Instantly share code, notes, and snippets.

@timvw
Last active October 17, 2019 09:04
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save timvw/ac6779ba20f20d44fd334d7e40326a6f to your computer and use it in GitHub Desktop.
Save timvw/ac6779ba20f20d44fd334d7e40326a6f to your computer and use it in GitHub Desktop.
connect to cloudkarafka with vanilla kafka tools
#!/bin/bash
# Export variables if not set already
export CLOUDKARAFKA_BROKERS="${CLOUDKARAFKA_BROKERS:-ark-01.srvs.cloudkafka.com:9094,ark-02.srvs.cloudkafka.com:9094,ark-03.srvs.cloudkafka.com:9094}"
export CLOUDKARAFKA_USERNAME="${CLOUDKARAFKA_USERNAME:-herecomesyourusername}"
export CLOUDKARAFKA_PASSWORD="${CLOUDKARAFKA_PASSWORD:-herecomesyourpassword}"
#!/bin/bash
# Load cloudkarafka variables (if not set already)
CLOUDKARAFKA_CONFIG_DIR=${CLOUDKARAFKA_CONFIG_DIR:-$(realpath "${0}" | xargs dirname)}
source "${CLOUDKARAFKA_CONFIG_DIR}/cloudkarafka_exports.sh"
# Verify that all required variables are set
[ -n "${KAFKA_HOME}" ] || (echo "KAFKA_HOME is not set."; exit 1)
[ -n "${CLOUDKARAFKA_BROKERS}" ] || (echo "CLOUDKARAFKA_BROKERS is not set."; exit 1)
[ -n "${CLOUDKARAFKA_USERNAME}" ] || (echo "CLOUDKARAFKA_USERNAME is not set."; exit 1)
[ -n "${CLOUDKARAFKA_PASSWORD}" ] || (echo "CLOUDKARAFKA_PASSWORD is not set."; exit 1)
# Build SASL config
export SASL_SAAS_CONFIG="org.apache.kafka.common.security.scram.ScramLoginModule required username=\"${CLOUDKARAFKA_USERNAME}\" password=\"${CLOUDKARAFKA_PASSWORD}\";"
$KAFKA_HOME/bin/kafka-console-producer.sh \
--producer-property security.protocol=SASL_SSL \
--producer-property sasl.mechanism=SCRAM-SHA-256 \
--producer-property sasl.jaas.config="${SASL_SAAS_CONFIG}" \
--broker-list "${CLOUDKARAFKA_BROKERS}" \
$@
#!/bin/bash
# Load cloudkarafka variables (if not set already)
CLOUDKARAFKA_CONFIG_DIR=${CLOUDKARAFKA_CONFIG_DIR:-$(realpath "${0}" | xargs dirname)}
source "${CLOUDKARAFKA_CONFIG_DIR}/cloudkarafka_exports.sh"
# Verify that all required variables are set
[ -n "${KAFKA_HOME}" ] || (echo "KAFKA_HOME is not set."; exit 1)
[ -n "${CLOUDKARAFKA_BROKERS}" ] || (echo "CLOUDKARAFKA_BROKERS is not set."; exit 1)
[ -n "${CLOUDKARAFKA_USERNAME}" ] || (echo "CLOUDKARAFKA_USERNAME is not set."; exit 1)
[ -n "${CLOUDKARAFKA_PASSWORD}" ] || (echo "CLOUDKARAFKA_PASSWORD is not set."; exit 1)
# Build SASL config
export SASL_SAAS_CONFIG="org.apache.kafka.common.security.scram.ScramLoginModule required username=\"${CLOUDKARAFKA_USERNAME}\" password=\"${CLOUDKARAFKA_PASSWORD}\";"
$KAFKA_HOME/bin/kafka-console-consumer.sh \
--consumer-property security.protocol=SASL_SSL \
--consumer-property sasl.mechanism=SCRAM-SHA-256 \
--consumer-property sasl.jaas.config="${SASL_SAAS_CONFIG}" \
--bootstrap-server "${CLOUDKARAFKA_BROKERS}" \
$@
@timvw
Copy link
Author

timvw commented Oct 17, 2019

Using these scripts you can now produce and consume from cloudkarafka

./cloudkarafka-console-producer.sh --topic my8ndu1i-test
./cloudkarafka-console-consumer.sh --topic my8ndu1i-test --from-beginning

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment