Skip to content

Instantly share code, notes, and snippets.

View tom-butler's full-sized avatar
🚵

tom-butler tom-butler

🚵
View GitHub Profile
@tom-butler
tom-butler / darwin-configuration.nix
Created August 18, 2023 10:17
nix-darwin configuration
{ config, pkgs, ... }:
{
security.pam.enableSudoTouchIdAuth = true;
networking.hostName = "mbp";
# List packages installed in system profile. To search by name, run:
# $ nix-env -qaP | grep wget
environment.systemPackages =
[
@tom-butler
tom-butler / k8s_roll_instances.sh
Created June 2, 2020 06:42
A shell script for replacing instances in an ASG whilst ensuring the deployments stay healthy.
#!/bin/bash
# set -e
# shellcheck disable=SC2178,SC2128
# time to wait for deployments to be healthy
wait_limit=${1:-900} # default set to 15 mins
# nodes to patch in a single go
max_nodes=${2:-50} # default set to max 50 nodes
@tom-butler
tom-butler / gid.sh
Created April 1, 2020 02:49
Finds missing gid and creates a new group for it
#!/bin/bash
# get error messages from groups
g=$(groups 2>&1 > /dev/null)
if [ -z "${g}" ]; then
# pull gid from error message
gid=${g/groups: cannot find name for group ID/}
# create a group with the missing gid
groupadd --gid "$gid" efs
fi
@tom-butler
tom-butler / Dask-Gateway.ipynb
Created October 1, 2019 05:45
dask-gateway
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@tom-butler
tom-butler / terraform.tfvars
Created September 26, 2019 09:36
Cluster config
# Cluster config
region = "ap-southeast-2"
owner = "deacepticons"
cluster_name = "dea-sandbox"
cluster_version = "1.13"
admin_access_CIDRs = {
"Everywhere" = "0.0.0.0/0"
}
users = [
"user/tbutler"
@tom-butler
tom-butler / local-k8s.sh
Created May 16, 2019 04:43
Local kubernetes cluster using k3d, with pvc support
#!/bin/bash
# Install k3d
k3d --version || wget -q -O - https://raw.githubusercontent.com/rancher/k3d/master/install.sh | bash
# verify it
k3d check-tools
# create a volume and cluster
docker volume create kube-volume
@tom-butler
tom-butler / local-kubernetes.sh
Last active January 12, 2020 22:20
local-kubernetes
#!/bin/bash
set -e
# Install k3d
wget -q -O - https://raw.githubusercontent.com/rancher/k3d/master/install.sh | bash
# verify it
k3d --version
k3d check-tools
@tom-butler
tom-butler / Orchestration.py
Created February 21, 2019 23:44
Infrastructure for watching SNS topic
import boto3
import click
import json
import schedule
from time import sleep
from hashlib import md5
from yaml import load
from functools import partial
from datetime import date, datetime, timedelta
@tom-butler
tom-butler / s3-downloader.py
Created November 6, 2018 00:43
Downloads Files from s3 using AWS Boto3
# coding: utf-8
import os
import boto3
from botocore.handlers import disable_signing
import logging
BUCKET_NAME='dea-public-data'
BUCKET_PATH='L2/sentinel-2-nrt/S2MSIARD/2018-07-30/S2B_OPER_MSI_ARD_TL_EPAE_20180730T055204_A007293_T51KVV_N02.06/'
def get_files(bucket_name, prefix):
@tom-butler
tom-butler / aws-config
Last active May 31, 2019 00:52
AWS CLI MFA
[default]
region = ap-southeast-2
[profile prod-admin]
source_profile = default
role_arn = arn:aws:iam::123456789012:role/write
mfa_serial = arn:aws:iam::0987654321098:mfa/uname
[profile prod-read]