Skip to content

Instantly share code, notes, and snippets.

Avatar
🐋

Ben Whaley bwhaley

🐋
View GitHub Profile
View .vimrc
set nocompatible
set hidden " Hide buffers rather than close them
set confirm
set encoding=utf-8
set showcmd " display incomplete commands
set number " show line numbers
set wrap linebreak nolist " wrap lines
set cursorline " highlight current row
set cursorcolumn " highlight current column
"set tags=tags
@bwhaley
bwhaley / aws_cleanup.py
Last active Feb 18, 2021
Quick and dirty script to clean up various types of resources in an AWS account
View aws_cleanup.py
#!/usr/bin/env python3
import argparse
import boto3
# Cleanup misc AWS resources
# NOTE: this script does not currently handle pagination
def get_regions():
ec2 = boto3.client("ec2")
View gist:7383803e1dec6bff2ea05d8af634d5cb
"""Put ECS Instances in DRAINING state so that all ECS Tasks running on them are migrated to other Instances.
Batches into chunks of 10 because of AWS api limitations (An error occurred InvalidParameterException when
calling the UpdateContainerInstancesState operation: instanceIds can have at most 10 items)
"""
def put_container_instances_in_draining_state(ecs_client, cluster_name, container_instance_arns):
batch_size = 10
n_batches = math.ceil(len(container_instance_arns)/batch_size)
for i in range(0, len(container_instance_arns), batch_size):
logger.info('Putting batch %d/%d of container instances %s in cluster %s into DRAINING state', i+1, n_batches, container_instance_arns, cluster_name)
ecs_client.update_container_instances_state(cluster=cluster_name, containerInstances=container_instance_arns[i:i + batch_size], status='DRAINING')
View TF trace
2019/10/21 09:34:02 [INFO] Terraform version: 0.12.12
2019/10/21 09:34:02 [INFO] Go runtime version: go1.12.9
2019/10/21 09:34:02 [INFO] CLI args: []string{"/usr/local/Cellar/tfenv/1.0.1/versions/0.12.12/terraform", "plan"}
2019/10/21 09:34:02 [DEBUG] Attempting to open CLI config file: /Users/bwhaley/.terraformrc
2019/10/21 09:34:02 [DEBUG] File doesn't exist, but doesn't need to. Ignoring.
2019/10/21 09:34:02 [INFO] CLI command args: []string{"plan"}
2019/10/21 09:34:02 [TRACE] Meta.Backend: no config given or present on disk, so returning nil config
2019/10/21 09:34:02 [TRACE] Meta.Backend: backend has not previously been initialized in this working directory
2019/10/21 09:34:02 [DEBUG] New state was assigned lineage "6ac92d61-1044-fec8-42c3-15ef1d2c34cc"
2019/10/21 09:34:02 [TRACE] Meta.Backend: using default local state only (no backend configuration, and no existing initialized backend)
@bwhaley
bwhaley / cloudwatch_log_subscriber.py
Created Jan 3, 2018
AWS lambda function to subscribe new CloudWatch Log groups to another lambda function
View cloudwatch_log_subscriber.py
# Lambda function to subscribe all new cloudwatch log groups to a log shipper function
# Used in conjunction with https://github.com/SumoLogic/sumologic-aws-lambda
import os
import logging
import json
import uuid
import boto3
from botocore.exceptions import ClientError
@bwhaley
bwhaley / cassandra-rolling-restart.yml
Created Oct 30, 2017
Ansible playbook for graceful rolling restart of the nodes in a cassandra cluster
View cassandra-rolling-restart.yml
- hosts: "*"
become: True
serial: 1
tasks:
- name: Initiate graceful shutdown
shell: "nodetool {{item}} && sleep 5"
with_items:
- disablethrift
- disablebinary
- disablegossip
View gist:9bafcbfce24c75a6eb022307930bd27b
sh ‘’’#!/bin/bash -l
cp do_image.txt pipeline/testing
cd pipeline/testing
terraform apply \\
-var do_image=\”\$(<do_image.txt)\” \\
-var do_token=\”\${DO_TOKEN}\” \\
-var ssh_fingerprint=\”\${SSH_FINGERPRINT}\”
terraform show terraform.tfstate \\
| grep ipv4_address | awk \”{print \$3}\” > ../../do_ip.txt
‘’’
@bwhaley
bwhaley / peer_routes.py
Last active Mar 7, 2020
A script to configure routes in in a VPC peering connection
View peer_routes.py
# Quick script to configure routes for a VPC peering connection
# Searches a region for all peering connection and prompts to choose one
# Configures routes between the peered networks for all routing tables
# STS/AssumeRole not implemented cross-account peering. Instead,
# Choose accepter/requestor depending on which credentials are set in the environment
# Enter either IPv4 and IPv6 route destinations
# Example usage:
# ( Assuming boto credentials are configured)
# $ pip install boto3
# $ python3.6 peer_routes.py
View kms_policy.json
{
"Version": "2012-10-17",
"Id": "cassandra-key-policy",
"Statement": [
{
"Sid": "Enable IAM User Permissions",
"Effect": "Allow",
"Principal": {
"AWS": "arn:aws:iam::09876512345:root"
},
@bwhaley
bwhaley / rds-logs.sh
Created Mar 18, 2016
bash script to read RDS logs
View rds-logs.sh
#!/usr/bin/env/bash
$db = $1
for logfile in $(aws rds describe-db-log-files --db-instance-identifier $db | jq '.DescribeDBLogFiles|.[]|.LogFileName')
do
aws rds download-db-log-file-portion --db-instance-identifier $db --log-file-name ${logfile} --starting-token 0 --output text >> $db-logs.txt
done