Skip to content

Instantly share code, notes, and snippets.

@RWJMurphy
RWJMurphy / access_s3_transformation
Last active October 11, 2015 09:28
Syncing and splunking Amazon S3 logs
^[[nspaces:s3_bucket_owner]]\s++[[nspaces:vhost]]\s++[[sbstring:req_time]]\s++[[nspaces:clientip]]\s++[[nspaces:user]]\s++[[nspaces:s3_request_id]]\s++[[nspaces:s3_operation]]\s++[[nspaces:s3_key]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:s3_error_code]]\s++[[nspaces:bytes]]\s++[[nspaces:s3_object_size]]\s++[[nspaces:s3_total_time]]\s++[[nspaces:s3_turnaround_time]]\s++"(?[[bc_domain:referer_]]?+[^"]*+)"\s++[[qstring:useragent]]\s++[[nspaces:s3_version_id]][[all:other]]
@RWJMurphy
RWJMurphy / are_my_damn_websites_alive.sh
Last active December 11, 2015 11:09
A fast, `parallel` powered website status checking script.
#!/bin/bash
check="./check_host.sh"
logfile="lastrun.log"
debug=$1
if command -v parallel >/dev/null 2>&1; then # PARALLEL :D
cat hosts | parallel --joblog $logfile -a - -j '1000%' "$check {} $debug"
else # NOT PARALLEL D:
rm -f $logfile && touch $logfile
@RWJMurphy
RWJMurphy / csv.py
Created February 4, 2013 01:37
A cut(1)-style tool for working with CSV files
#!/usr/bin/env python
from __future__ import print_function
import argparse
import csv
import os
import sys
def fields(string):
parsed_fields = []
@RWJMurphy
RWJMurphy / hsort.py
Created February 4, 2013 01:38
Like sort(1), only in Python, and capable of sorting by line length.
#!/usr/bin/env python
import argparse
import os
import random
import re
import string
import sys
def number_or_zero(x):
@RWJMurphy
RWJMurphy / markov.py
Created February 4, 2013 01:39
Markov chain text generator in python
#!/usr/bin/env python
from __future__ import print_function
import argparse
from collections import defaultdict
import os
import random
import sys
def main(args):
@RWJMurphy
RWJMurphy / random_passphrase_generator.py
Last active December 12, 2015 03:08
Does what it says on the box. Inspired by http://xkcd.com/936/
#!/usr/bin/env python
from __future__ import print_function
import argparse
import math
import random
import string
import sys
class PassGen:
def __init__(self, dictionary, connectives, minimum_word_length, no_white_space=False):
@RWJMurphy
RWJMurphy / ts
Created February 4, 2013 01:40
Reads lines from stdin, appends a timestamp and prints to stdout
#!/usr/bin/env python
from __future__ import print_function
from datetime import datetime
import sys
def main():
try:
while 1:
line = sys.stdin.readline()
@RWJMurphy
RWJMurphy / cli_stub.py
Last active December 12, 2015 03:08
PEP8 compliant stub for building Python CLI tools
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# vim: ft=python ts=4 sw=4 expandtab
"""A stub for command line applications in Python."""
__version__ = "0.0.0"
import argparse
import os
@RWJMurphy
RWJMurphy / mcd.sh
Created February 4, 2013 01:45
magic cd
function mcd() {
cd=$(echo $1 | sed 's/\(.\)/\/\1*/g')/
dirs=`\ls -1d $cd 2>/dev/null`
dira=($dirs)
dircount=${#dira[*]}
if [[ $dircount == 0 ]]; then
echo "No match for ${cd}"
elif [[ $dircount == 1 ]]; then
echo cd ${dirs}
cd "${dirs}"
reedm@www:~/tmp $ ls -ld .
drwx------ 2 reedm reedm 4096 Feb 4 14:54 ./
reedm@www:~/tmp $ cat > foo
This is a file
reedm@www:~/tmp $ chmod -w foo
reedm@www:~/tmp $ mv foo bar
reedm@www:~/tmp $ ls -la
total 12
drwx------ 2 reedm reedm 4096 Feb 4 14:54 ./
drwx-----x 54 reedm reedm 4096 Feb 4 10:18 ../