Skip to content

Instantly share code, notes, and snippets.

@AlexJReid
AlexJReid / Dockerfile
Last active February 18, 2023 14:26
skeleton
FROM gcr.io/distroless/base
COPY ./dist/linux/amd64/changeme /changeme
ENTRYPOINT ["/changeme"]
@AlexJReid
AlexJReid / enable-google-extensions.sh
Created April 16, 2020 23:16
Cloud Dataproc and Druid - initialisation action to enable batch ingest from GCS
#!/bin/bash
set -euxo pipefail
# Override loaded extensions
cat << EOF >> /usr/lib/druid/conf/druid/_common/common.runtime.properties
# Override
druid.extensions.loadList=["druid-histogram", "druid-datasketches", "druid-lookups-cached-global", "mysql-metadata-storage", "druid-hdfs-storage", "druid-google-extensions"]
EOF
# Restart
@AlexJReid
AlexJReid / enable-druid-router.sh
Created April 16, 2020 23:15
Cloud Dataproc and Druid - initialisation action to enable the Druid router
#!/bin/bash
set -euxo pipefail
# Add router config
mkdir -p /usr/lib/druid/conf/druid/router/
cat << EOF > /usr/lib/druid/conf/druid/router/runtime.properties
druid.service=druid/router
druid.port=8888
druid.processing.numThreads=1
druid.processing.buffer.sizeBytes=1000000
{
"stage": "train-display-board",
"data": {
"refresh_rate": 30,
"origin": "NCL",
"response_time": "2019-12-11",
"valid_for": 60,
"departures": [
{"departure_time": "14:25", "destination": "Edinburgh Waverley", "platform": "Plat 2", "status": "On time", "calling_at": ["Dunbar"]},
{"departure_time": "14:25", "destination": "Metro Centre", "platform": "Plat 2", "status": "On time", "calling_at": ["Dunbar"]},
{
"stage": "metrics",
"metrics": [
{"value": "7", "label": "Days to go"},
{"value": "2", "label": "Naughty"},
{"value": "312", "label": "Presents"},
{"value": "9", "label": "Reindeer"}
]
}
@AlexJReid
AlexJReid / ProtobufBinaryMessageWriter.java
Last active February 8, 2023 19:42
Jersey/JAX-RS protobuf produce and consume
package com.github.alexjreid;
import com.google.protobuf.Message;
import javax.ws.rs.Produces;
import javax.ws.rs.WebApplicationException;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.MultivaluedMap;
import javax.ws.rs.ext.MessageBodyWriter;
import javax.ws.rs.ext.Provider;
AWSDynamoDBAttributeValue *dbav = [AWSDynamoDBAttributeValue new];
dbav.S = _userID;
update.key = @{@"UserID": dbav};
// can be boiled down to:
update.key = @{@"UserID": [AWSDynamoDBAttributeValue withString:_userID]};
/* ----------------- */
@AlexJReid
AlexJReid / web.js
Created January 5, 2012 21:08
parallel requests to SimpleDB in node
var express = require('express'),
simpledb = require('simpledb'),
Step = require('step'),
sdb = new simpledb.SimpleDB({
keyid: 'your aws key',
secret: 'your aws secret'
}),
app = express.createServer(express.logger()),