Skip to content

Instantly share code, notes, and snippets.

@AlexJReid
Created April 16, 2020 23:15
Show Gist options
  • Save AlexJReid/1f0cc59d31a2ac8ff1e69d2c756a098c to your computer and use it in GitHub Desktop.
Save AlexJReid/1f0cc59d31a2ac8ff1e69d2c756a098c to your computer and use it in GitHub Desktop.
Cloud Dataproc and Druid - initialisation action to enable the Druid router
#!/bin/bash
set -euxo pipefail
# Add router config
mkdir -p /usr/lib/druid/conf/druid/router/
cat << EOF > /usr/lib/druid/conf/druid/router/runtime.properties
druid.service=druid/router
druid.port=8888
druid.processing.numThreads=1
druid.processing.buffer.sizeBytes=1000000
druid.router.defaultBrokerServiceName=druid/broker
druid.router.coordinatorServiceName=druid/coordinator
druid.router.http.numConnections=50
druid.router.http.readTimeout=PT5M
druid.router.http.numMaxThreads=100
druid.server.http.numThreads=100
druid.router.managementProxy.enabled=true
EOF
# Install service
cat << EOF > /etc/systemd/system/druid-router.service
[Unit]
Description=Router
Wants=network-online.target
After=network-online.target
[Service]
WorkingDirectory=/usr/lib/druid
Environment="JAVA_HOME=/usr/lib/jvm/adoptopenjdk-8-hotspot-amd64"
Environment="HADOOP_CONF_DIR="
User=root
Group=root
Type=simple
ExecStart=/bin/bash -c "java -server -Xms128m -Xmx128m -XX:MaxDirectMemorySize=512m -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/tmp -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -cp "/usr/lib/druid/conf/druid/_common:/usr/lib/druid/lib/*:/usr/lib/druid/conf/druid/router" org.apache.druid.cli.Main server router > /var/log/druid/router.log"
[Install]
WantedBy=multi-user.target
EOF
# Enable and start router service
ROLE=$(/usr/share/google/get_metadata_value attributes/dataproc-role)
if [[ $ROLE == 'Master' ]]; then
echo "Starting Druid router..."
systemctl daemon-reload
systemctl start druid-router
systemctl enable druid-router
fi
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment