Skip to content

Instantly share code, notes, and snippets.

View chrisdpa-tvx's full-sized avatar

Chris Atkinson chrisdpa-tvx

View GitHub Profile
set -exu
SPOTFLEETREQ=$1
CLUSTER=$2
AZ=$3
fleet_sz=$(aws ec2 describe-spot-fleet-instances --spot-fleet-request-id ${SPOTFLEETREQ} | jq -r '.ActiveInstances|length')
container_sz=$(aws ecs list-container-instances --cluster ${CLUSTER} | jq -r '.containerInstanceArns[]' | \
xargs -n10 -J % aws ecs describe-container-instances --cluster ${CLUSTER} --container-instances % | \
jq -r ".containerInstances[].attributes[]|select(.name==\"ecs.availability-zone\" and .value==\"${AZ}\")|.value"|wc -l | xargs)
@chrisdpa-tvx
chrisdpa-tvx / cluster_images.sh
Created July 26, 2017 14:00
Drill to docker image from a cluster
CLUSTER=$1
aws ecs list-services --cluster ${CLUSTER} | jq -r '.[] | .[]' | \
xargs -J % -n10 aws ecs describe-services --cluster ${CLUSTER} --services % | jq -r '.services[] | .serviceArn' | \
xargs -J % -n10 aws ecs describe-services --cluster ${CLUSTER} --services % | jq -r '.services[]|.taskDefinition' | \
xargs -J % -n1 aws ecs describe-task-definition --task-definition % | jq -r '.taskDefinition|.containerDefinitions[]|.image'
@chrisdpa-tvx
chrisdpa-tvx / ecs_sad.sh
Last active July 24, 2017 09:38
Find Unhappy services
CLUSTER=$1
sts aws ecs list-services --cluster ${CLUSTER} |\
jq -r '.[] | .[]' | xargs -J % -n10 sts \
aws ecs describe-services --cluster ${CLUSTER} \
--services % | jq -r '.services[] | select(.desiredCount!=.runningCount) | .serviceArn' > services
while read r;do
sts aws ecs update-service \
--cluster ${CLUSTER} \
--service ${r} \
See netstat detail of a running docker container:
nsenter -t $(docker inspect -f '{{.State.Pid}}' _container_id_) -n netstat -tunapp
@chrisdpa-tvx
chrisdpa-tvx / athena.rst
Last active March 20, 2022 06:28
Create an Athena database, table, and query

All Your Data Does Not Belong In a Database

Businesses are machines producing mountains of data about sales, usage, customer, costs, etc... Traditionally data processing is highly centralised with teams of staff and computer running hot a whirling ready to process. We can do better than moving the mountain of data into the corporate data machine - so long as that machinary is light enough to be moved to the data.

Don't move the mountain - Bring the processing to the data

We've had this problem; a huge directory of files in CSV format, conataining vital information for our business. But it's in CSV, requires analysis, and don't you don't feel like learning sed/grep/awk today - besides it's 2017 and no-one thinks those tools are easy to use.

@chrisdpa-tvx
chrisdpa-tvx / gnupg.py
Created May 31, 2017 10:57
Encrypt+Sign then decrypt using gpg
import gnupg
import cStringIO
gpg = gnupg.GPG()
gpg.encoding = 'utf-8'
me = {'name_real': 'alice',
'name_email': 'alice@example.com',
'expire_date': '2024-04-01',
'passphrase': 'alicespassword'}
@chrisdpa-tvx
chrisdpa-tvx / dos2unix.md
Last active May 23, 2017 15:28
Convert between UNIX and DOS line endings