gordonmurray

Using Skeema to create or alter database tables

In this example we will use Skeema to create and alter some tables in a MariaDB database.

Skeema is a very useful tool that works with mySQL or Mariadb databases to help you to compare your development database files to a production database. Skeema will generate any alter statements that are required and apply those changes to your production database.

Skeema is a free and open source tool developed using Go and is available on Github at https://github.com/skeema/skeema

If you have very large tables in your production database, performing alters to the tables can take a lot of time to complete fully and may cause problems for your users if there are users actively using your software while you change the database.

gordonmurray

#!/usr/bin/env bash

set -xe

REGION="eu-west-1"

# list RDS instances to a file
aws rds --region ${REGION} describe-db-instances | jq -r '.DBInstances[].DBInstanceIdentifier' > rds.log

# loop over the rds instances in the file

gordonmurray

# Simple script to list all RDS instances and then disable Automatic Minor Version Upgrades 

# list RDS instances to a file
aws rds --region eu-west-1 describe-db-instances | jq -r '.DBInstances[].DBInstanceIdentifier' > rds.log

# loop over the items in the file, disable auto minor version upgrades
while read rds; do
    aws rds modify-db-instance --region eu-west-1 --db-instance-identifier ${rds} --no-auto-minor-version-upgrade
    sleep 5
done <rds.log

gordonmurray

# Moved to a Repo here instead and added some additional files
https://github.com/gordonmurray/ecs_cluster_using_aws_cli

gordonmurray

#!/usr/bin/env bash 

# The following assumes you have an AWS account with the AWS CLI installed locally
# It will ask which VPC and Subnets to use

# Show the commands being executed
set -ex

# AWS region
REGION="eu-west-1"

gordonmurray

Convert Caddy Server certificates to LetsEncrypt certificates to be used by Nginx

Caddy

When using Caddy Server, it stores certificates in ~/.caddy/acme/acme-v01.api.letsencrypt.org/sites/{your domain name}/

3 files are stored in the folder called:

• {yourdomain}.crt
• {yourdomain}.json
• {yourdomain}.key

gordonmurray

# prerequisites for docker
apt-get update
apt-get -y install apt-transport-https
apt-get -y install ca-certificates
apt-get -y install curl
apt-get -y install software-properties-common

# docker repos
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - \
    && echo "deb [arch=amd64] https://download.docker.com/linux/ubuntu xenial stable" >> /etc/apt/sources.list.d/additional-repositories.list \

gordonmurray

# list your aws elasticache parameter groups
aws elasticache describe-cache-parameter-groups --profile=xxxxx --region=us-east-1

# list the parameters currently set in a particular aws elasticache parameter groups
aws elasticache describe-cache-parameters --profile=xxxxx --region=us-east-1 --cache-parameter-group-name=xxxxx > xxxxx.json

gordonmurray

#!/usr/bin/env bash

# white list all public IPs from an AWS region in to a security group

security_group="sg-xxxxxx"
region="eu-west-1"
port="3306"

# list the public IPs from the instances in region
public_ips=`aws ec2 describe-instances --query "Reservations[*].Instances[*].PublicIpAddress" --output=text --region=${region} | xargs`

gordonmurray

#!/usr/bin/env bash

# A small script to take a larger text based file and break it down to several smaller files
# of 500 lines each for easier processing, keeping the headings at the top of each file

FILENAME=${1}

if [ -z ${FILENAME} ]; then
  echo "Please provide a filename, for example: bash split.sh /path/to/data.csv";
else