Skip to content

Instantly share code, notes, and snippets.

@AlexJReid
Created April 16, 2020 23:16
  • Star 1 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save AlexJReid/1684106b2118dfa69f98b68345cad634 to your computer and use it in GitHub Desktop.
Cloud Dataproc and Druid - initialisation action to enable batch ingest from GCS
#!/bin/bash
set -euxo pipefail
# Override loaded extensions
cat << EOF >> /usr/lib/druid/conf/druid/_common/common.runtime.properties
# Override
druid.extensions.loadList=["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "mysql-metadata-storage", "druid-hdfs-storage", "druid-google-extensions"]
EOF
# Restart
ROLE=$(/usr/share/google/get_metadata_value attributes/dataproc-role)
if [[ $ROLE == 'Master' ]]; then
systemctl restart druid-router
systemctl restart druid-coordinator
systemctl restart druid-overlord
systemctl restart druid-broker
else
systemctl restart druid-historical
systemctl restart druid-middle-manager
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment