Skip to content

Instantly share code, notes, and snippets.

View lukeplausin's full-sized avatar

Luke Plausin lukeplausin

View GitHub Profile
@lukeplausin
lukeplausin / creation_date_s3_tags.py
Created December 31, 2016 16:07
Adds tags for creation date to objects in an S3 bucket. Useful if you've just used `aws s3 sync` and want to add metadata.
from __future__ import print_function
import os
import sys
from datetime import datetime
import boto3
walk_dir = "/Volumes/MyFiles"
client = boto3.client('s3')
bucket = "MyBucket"
@lukeplausin
lukeplausin / bash_aws_jq_cheatsheet.sh
Last active August 29, 2024 20:27
AWS, JQ and bash command cheat sheet. How to query, cut and munge things in JSON generally.
# Count total EBS based storage in AWS
aws ec2 describe-volumes | jq "[.Volumes[].Size] | add"
# Count total EBS storage with a tag filter
aws ec2 describe-volumes --filters "Name=tag:Name,Values=CloudEndure Volume qjenc" | jq "[.Volumes[].Size] | add"
# Describe instances concisely
aws ec2 describe-instances | jq '[.Reservations | .[] | .Instances | .[] | {InstanceId: .InstanceId, State: .State, SubnetId: .SubnetId, VpcId: .VpcId, Name: (.Tags[]|select(.Key=="Name")|.Value)}]'
# Wait until $instance_id is running and then immediately stop it again
aws ec2 wait instance-running --instance-id $instance_id && aws ec2 stop-instances --instance-id $instance_id
# Get 10th instance in the account
@lukeplausin
lukeplausin / dfsr-tailer.ps1
Created January 19, 2017 10:03
What is DFSR doing? Short script to tail the DFS-R logs in windows debug log.
$WorkingDir = "$HOME/Documents/dfsr-logs"
new-alias 7z "C:\Program Files\7-zip\7z.exe"
cp C:/Windows/debug/Dfsr00999.log.gz $WorkingDir
rm -recurse -force $WorkingDir/extracted
7z e $WorkingDir/Dfsr00999.log.gz "-o$WorkingDir\extracted"
# gc $workingdir\extracted\Dfsr00999.log | sls "^[^+]"
echo "Last 60 lines of DFSR log"
gc $workingdir/extracted/Dfsr00999.log -tail 60
@lukeplausin
lukeplausin / install-software.ps1
Last active June 4, 2019 21:00
Script to get you up and running a bit quicker after reinstalling windows
# You will need to run this in the shell first...
Set-ExecutionPolicy Unrestricted
# Install chocolatey
iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex
# Install my selection of useful software
choco install googlechrome flashplayerplugin javaruntime adobereader 7zip.install atom procexp sysinternals curl mRemoteNG gimp vim python2 python windirstat libreoffice partitionwizard skype macrium
# Finish off the installation of Macrium
(Invoke-Command "C:\tools\ReflectDL.exe") -and (rm "C:\tools\ReflectDL.exe")
# Add a local user
@lukeplausin
lukeplausin / pre-push
Last active November 24, 2022 12:06
Git hook to push to a second remote if pushing to master
#!/bin/sh
# An example hook script to verify what is about to be pushed. Called by "git
# push" after it has checked the remote status, but before anything has been
# pushed. If this script exits with a non-zero status nothing will be pushed.
#
# This hook is called with the following parameters:
#
# $1 -- Name of the remote to which the push is being done
# $2 -- URL to which the push is being done
@lukeplausin
lukeplausin / aws-depaginate-boto.py
Created April 19, 2017 16:12
Example for depaginating boto3 results from AWS.
import boto3
def depaginate(function, resource_key, **kwargs):
# Will depaginate results made to an aws client
response = function(**kwargs)
results = response[resource_key]
while (response.get("NextToken", None) is not None):
response = function(NextToken=response.get("NextToken"), **kwargs)
results = results + response[resource_key]
return results
@lukeplausin
lukeplausin / awscm.sh
Created July 17, 2017 12:59
awscm (simple credential manager script for AWS, in bash)
#!/bin/bash
#
# AWSCM: Command line tool for quickly switching between AWS profiles.
# To install:
# mkdir ~/.awscm/
# curl $filepath > ~/.awscm/awscm.sh
# echo 'source ~/.awscm/awscm.sh' >> ~/.bashrc
# echo 'source ~/.awscm/awscm.sh' >> ~/.zshrc
OUTPUT_FORMATS=(
@lukeplausin
lukeplausin / jaeger_log_forwarder.py
Last active March 20, 2023 13:31
Python custom logger - forward log events to Jaeger
import logging
import time
import yaml
from jaeger_client import Config
import os
import datetime
import logging
from logging import StreamHandler
@lukeplausin
lukeplausin / load_any_yaml_tag.py
Last active July 22, 2023 09:01
Load any YAML tag including custom tags in python with PyYAML
# Inspired by this great answer...
# https://stackoverflow.com/a/60785401
import yaml
class SafeUnknownConstructor(yaml.constructor.SafeConstructor):
def __init__(self):
yaml.constructor.SafeConstructor.__init__(self)
def construct_undefined(self, node):
@lukeplausin
lukeplausin / easy_ssl_certificates
Last active June 17, 2020 22:45
Easy SSL certificates - create your own CA in seconds!
######################
# My CA - Create your own certificate authority!
# for people who need lots of certificates...
#
# Create a workspace and save this Makefile in the workspace
# $ mkdir ~/.ssl/
# $ curl https://gist.githubusercontent.com/lukeplausin/b1e78b3b55490d91997bcb13532ce663/raw > ~/.ssl/Makefile
# $ cd ~/.ssl/
#
# $ make ca # Generate the certificate authority