pwillis-els

node('node-label') {
    stage('Checkout') {
        scm checkout
    }

    stage('Tag') {
        sh 'git tag my-tag'
        sh 'sh git-credential-env.sh --install'

        withCredentials([[

pwillis-els

#!/bin/sh
# Runs a detached Docker container with an entrypoint to run an sshd daemon.

CONTAINER_IMG="my-container-image:latest"
CONTAINER_USER="deploy" # the user the container runs as
CONTAINER_HOME="/home/$CONTAINER_USER" # home directory in the container
HOST_SSH_PORT="2222" # The port to export sshd to on localhost
CONTAINER_SSH_PORT="2222" # The port of sshd in the container

set -eu

pwillis-els

#!/bin/sh
set -eu
function json_v () { python -c 'import sys,json;j=json.load(sys.stdin); a=sys.argv[1:]; print("\n".join([j[k] for k in a]))' "$@" ; }
function _ec2_credentials () {
    JSON=`curl -s http://169.254.169.254/latest/meta-data/identity-credentials/ec2/security-credentials/ec2-instance`
    if [ -n "$JSON" ] ; then
        AWS_ACCESS_KEY_ID=`printf "%s" "$JSON" | json_v AccessKeyId`
        AWS_SECRET_ACCESS_KEY=`printf "%s" "$JSON" | json_v SecretAccessKey`
        AWS_SESSION_TOKEN=`printf "%s" "$JSON" | json_v Token`
        export AWS_ACCESS_KEY_ID AWS_SECRET_ACCESS_KEY AWS_SESSION_TOKEN

pwillis-els

#!/bin/sh
python -c 'import sys,json;j=json.load(sys.stdin); a=sys.argv[1:]; print("\n".join([j[k] for k in a]))' "$@"

pwillis-els

#!/bin/sh
python -c 'import sys,json;j=json.load(sys.stdin); print( "\n".join(["%s=%s" % (k, j[k]) for k in j]) )'

pwillis-els

Optimizing S3 file transfer speed

There is a nifty tool https://github.com/larrabee/s3sync which can transfer files very quickly with S3, but it reads all files into memory. If you're transferring lots of large files this process will quickly get killed by the kernel due to OOM.

The next best thing I have found is to tweak the AWS CLI to use aws s3 sync as fast as possible.

# ~/.aws/config
[default]

pwillis-els

#!/bin/sh
set -e
# commands taken from https://docs.docker.com/engine/install/ubuntu/

sudo apt-get remove docker docker-engine docker.io containerd runc || true

sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \

pwillis-els

Heap dumping a Java process running in AWS ECS

Note: this guide is designed for AWS ECS services, but starting from Step 4 is functionally equivalent to any Docker container on a Linux host.

Step 1. Look up ECS service

1. Log into the AWS Console using the appropriate AWS account
2. Navigate to AWS ECS service clusters (https://console.aws.amazon.com/ecs/home)
3. Make sure you are in the correct region, if not, switch to the correct region (second drop-down menu in top right corner)
4. Select the correct cluster (ex: https://console.aws.amazon.com/ecs/home?region=us-east-1#/clusters//services)
5. In the Services tab, In the 'Filter in this page' text box, type the name of the service

pwillis-els

Bash Cheat Sheet

Strings

Check if a substring exists in a string

name="somethinglong"
substr="thing"
if [ ! "${name/$substr/}" = "$name" ] ; then
    echo "'$substr' found in '$name'"

pwillis-els

(this can probably be greatly improved...)

Run the following command to enter a Docker container with a Go environment set up:

$ docker run --rm -it -v "$PWD":/usr/src/myapp -w /usr/src/myapp golang:1.8 bash

Inside the container, run the 'go get' command for the software you want to build and install:

root@3aac13b2bce8:/usr/src/myapp# go get github.com/itchio/gothub