Skip to content

Instantly share code, notes, and snippets.

View gaving's full-sized avatar
🎯
Focusing

Gavin Gilmour gaving

🎯
Focusing
View GitHub Profile
@pvillard31
pvillard31 / MDD.xml
Created August 29, 2018 16:15
Template for Monitoring Driven Development in NiFi
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<template encoding-version="1.2">
<description></description>
<groupId>8927f4c0-0160-1000-597a-ea764ccd81a7</groupId>
<name>MDD</name>
<snippet>
<connections>
<id>a2098494-cce9-3fa4-0000-000000000000</id>
<parentGroupId>a8352767-434f-3321-0000-000000000000</parentGroupId>
<backPressureDataSizeThreshold>1 GB</backPressureDataSizeThreshold>
@Ocramius
Ocramius / Caddyfile
Last active March 11, 2024 22:14
Example docker + docker-compose + caddy + traefik setup that routes to multiple apps from one exposed HTTP port
:80 {
root /serve
}
@AntonFriberg
AntonFriberg / Dockerfile
Last active July 10, 2023 08:50
Simple docker cron sidecar container based on alpine image
FROM alpine:3.8
# Alpine comes with built in cron schedules
# min hour day month weekday command
# */15 * * * * run-parts /etc/periodic/15min
# 0 * * * * run-parts /etc/periodic/hourly
# 0 2 * * * run-parts /etc/periodic/daily
# 0 3 * * 6 run-parts /etc/periodic/weekly
# 0 5 1 * * run-parts /etc/periodic/monthly
@jamiejackson
jamiejackson / README.md
Last active June 8, 2023 16:27
Upgrade a Docker Secret in a Running Swarm Service

Update: I'm using this strategy, at the moment: https://github.com/jamiejackson/docker-secret-hash


This is a helper script for updating a secret in a running service. It does cause the service containers to restart (there's no avoiding that), but it does so as elegantly/conveniently as I know how.

See the motivation for this script, as well as alternatives, here.

Caveat:

  • Secret names with a length approaching 64 characters might exhibit weird behavior, as this relies on a version token being tacked onto the secret name. Since the secret name is truncated to 64 characters, a long name might result in a smaller (or nonexistent) version token, leading to uniqueness problems and unexpected behavior.

gif-from-tweet

There are so many great GIFs out there and I want to have copies of them. Twitter makes that harder than it should be by converting them to MP4 and not providing access to the source material. To make it easier, I made a bash pipeline that takes a tweet URL and a filename, extracts the MP4 from that tweet and uses ffmpeg to convert back to GIF.

Dependencies

  • ffmpeg
    • macOS: brew install ffmpeg
    • Ubuntu/Debian: apt install ffmpeg
@swalkinshaw
swalkinshaw / tutorial.md
Last active November 13, 2023 08:40
Designing a GraphQL API
@Aghassi
Aghassi / docker-compose.yml
Last active April 1, 2024 21:03
LinuxServer Docker Compose: Plex, Sonarr, Radarr, NZBGet, Let's Encrypt, Time Machine
version: '2'
services:
plex:
image: linuxserver/plex
container_name: plex
volumes:
- /path/to/plex/config:/config
- /path/to/plex/Movies:/data/movies
- /path/to/plex/Shows:/data/tvshows
- /path/to/plex/transcode:/data/transcode
@ijokarumawak
ijokarumawak / 0.JoinCSVRecords.md
Last active August 31, 2023 06:13
A NiFi example flow which joins two CSV files using LookupRecord processor.

Join, Enrich multiple columns by looking up an external CSV file

This example illustrates how to use LookupRecord processor to join multiple columns from another CSV file. The key configuration is to define a result schema at 'RecordWriter' having the original columns AND the columns those are enriched by the lookuped values. Also, use 'Insert Record Fields' as LookupRecord 'Record Result Content', and '/' as 'Result Record Path' to join the lookuped values into the original record.

Given file-1.csv to lookup file-2.csv, this example outputs folloing content as a result:

Emp_Id,Name,Address,Mobile_No,Salary,Department
@jxub
jxub / pandas_labeled_csv_import_to_mongo.py
Created September 22, 2017 11:51
A simple mongoimport for importing csv files with python and pymongo
import pandas as pd
from pymongo import MongoClient
import json
def mongoimport(csv_path, db_name, coll_name, db_url='localhost', db_port=27000)
""" Imports a csv file at path csv_name to a mongo colection
returns: count of the documants in the new collection
"""
client = MongoClient(db_url, db_port)
db = client[db_name]
@ijokarumawak
ijokarumawak / 0.flow.png
Last active February 15, 2022 11:05
NiFi example flow which limits number of incoming FlowFiles by Wait and Notify.
0.flow.png