Skip to content

Instantly share code, notes, and snippets.

@erincerys
erincerys / table-rotation.sql
Last active August 29, 2015 13:56
Rotate MySQL table data within the data store selectively, using a date range on a column
-- If your table is really large, and it very well may be if you've the need to rotate it, you'll want to make sure the column that holds the timestamp/date is indexed.
-- WARNING: This script incurs a write lock for the duration of the trigger recreation and table rename. If your application can't handle this, you might lose data!
-- Create our new skeleton table
CREATE TABLE x_copy LIKE x;
-- Replace the interval with the timespan of data you'd like to keep, or
-- replace this statement with one that stores an id in the local variable of which all higher will be retained
SELECT MIN(id) INTO @end_id FROM x WHERE created_at >= DATE(NOW()) - INTERVAL 3 WEEK;
@erincerys
erincerys / spinhours.sh
Last active December 30, 2015 07:59
Return your drives' power on hours via smartctl (requires smartmontools)
#!/bin/bash
if [ ! "$(which smartctl)" ] ; then
echo "smartmontools is not installed"
exit 1
fi
logical=($(df | grep -Eo 'sd[a-z]'))
# This doesn't preserve the mountpoint of the raid device, but whatevs for now
@erincerys
erincerys / deduplicate-column.sql
Last active December 27, 2015 15:09
If you require adding a UNIQUE key to a MySQL table, but the desired column contains duplicates, this stored procedure will assist in raising cardinality by appending random numbers to the end of a duplicate record.
DELIMITER $$
CREATE PROCEDURE `sp_utility_deduplicate_column` (IN table_in VARCHAR(64), IN unique_in VARCHAR(64), IN nonunique_in VARCHAR(64), length_in INT(11))
BEGIN
/* sp_utility_deduplicate_column
IN table_in The name of the table to execute deduplication on
IN unique_in The PRIMARY KEY for table_in (auto-incremented integer recommended)
IN nonunique_in The column that contains duplicated values (INDEX recommended)
@erincerys
erincerys / fscheck.sh
Last active December 26, 2015 14:29
Shutdown a server if a fusefs connection become unreliable
#!/bin/bash
# If a fusefs connection has been lost, shutdown the server and optionally log the event to an S3 object.
# I use this AWS EC2 instances in auto-scaling groups. If it shuts down, it gets terminated by the ASG, and replaced with a fresh one.
# Might be some fringe cases in the error handling code that haven't been encountered or thought of. Hence, might not be 100% reliable, so use at your own risk
cd /root
# Create lock
lockfile='/root/fscheck.lock'
@erincerys
erincerys / mysql2file.py
Created September 26, 2013 16:59
Execute mysql query and write to file utilizing a streaming cursor
#! /usr/bin/env python
from sys import argv
import csv
import MySQLdb
from MySQLdb import cursors
mysql_params = [argv[1], argv[2], argv[3], argv[4], argv[5]]
output_file = argv[6]
mysql_query = argv[7]
@erincerys
erincerys / rds2redshift.sh
Last active January 27, 2020 15:38
Loads a MySQL data dump into a Redshift table. Useful for AWS RDS instances. Dependent on external script to stream MySQL data to file, and postgres psql command line client.
PYTHON_PATH=$(which python)
PSQL_PATH=$(which psql)
MYSQL_SCRIPT='mysql2file.py'
MYSQL_SERVER=
MYSQL_PORT=3306
MYSQL_DATABASE=
MYSQL_USER=
MYSQL_PASSWORD=
@erincerys
erincerys / atlassian2s3.sh
Created September 26, 2013 16:45
Schedules a backup of an Atlassian OnDemand instance and uploads the resulting file to S3
#!/bin/bash
#
# Obtained from:
# https://confluence.atlassian.com/display/ONDEMANDKB/Automatic+backups+for+JIRA+OnDemand
# and modified to include:
# - longer waiting period for backup to be created
# - S3 upload
#
@erincerys
erincerys / dumper.pl
Last active August 10, 2016 14:56 — forked from shantanuo/dumper.pl
MySQLdump filter -- Removes or replaces the DEFINER clauses from a dump
#!/usr/bin/perl
use strict;
use warnings;
use Getopt::Long qw(:config no_ignore_case );
my $replace = undef;
my $delete = undef;
my $help = 0;
GetOptions (