Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Cloud Dataproc and Druid - initialisation action to enable batch ingest from GCS
#!/bin/bash
set -euxo pipefail
# Override loaded extensions
cat << EOF >> /usr/lib/druid/conf/druid/_common/common.runtime.properties
# Override
druid.extensions.loadList=["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "mysql-metadata-storage", "druid-hdfs-storage", "druid-google-extensions"]
EOF
# Restart
ROLE=$(/usr/share/google/get_metadata_value attributes/dataproc-role)
if [[ $ROLE == 'Master' ]]; then
systemctl restart druid-router
systemctl restart druid-coordinator
systemctl restart druid-overlord
systemctl restart druid-broker
else
systemctl restart druid-historical
systemctl restart druid-middle-manager
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment