terabyte

Prologue

I wrote this answer on stackexchange, here: https://stackoverflow.com/posts/12597919/

It was wrongly deleted for containing "proprietary information" years later. I think that's bullshit so I am posting it here. Come at me.

The Question

Amazon is a SOA system with 100s of services (or so says Amazon Chief Technology Officer Werner Vogels). How do they handle build and release?

terabyte

def default_serialize_func(o):
    """
    Use like this: logging.debug(f"print this object: {json.dumps(myobject, indent=4, sort_keys=True, default=default_serialize_func)}")
    """
    if hasattr(o, '__dict__'):
        return o.__dict__
    return f"<could not serialize {o.__class__}>"

terabyte

#!/bin/bash

set -e

GIT_DIR=$HOME/projects/private_notes

# only do it if we have network
if ping -c1 -w5 -n 8.8.8.8 | grep -q '1 received'; then

    # first fetch updates

terabyte

#!/bin/bash

set -e

GIT_DIR=$HOME/projects/private_notes

# first fetch updates
git -C $GIT_DIR fetch -q origin > /dev/null

if [[ "x$(git -C $GIT_DIR status --porcelain)" == "x" ]]; then

terabyte

#!/bin/bash

set -e

GIT_DIR=$HOME/projects/private_notes

git -C $GIT_DIR add -A
git -C $GIT_DIR commit -q -am "update from '$(hostname)' on $(date '+%Y%m%d %H:%M %Z')" > /dev/null || exit 0

terabyte

#!/usr/bin/env bash
# Copyright (c) 2017 Cloudera, Inc. All rights reserved.

set -exo pipefail

# Usage: ./docker.sh
# Usage: ./docker.sh test/test.sh
# Usage: INTERACTIVE=1 ./docker.sh

# This script invokes the docker container, does the necessary contortions to

terabyte

FROM debian:11


RUN apt-get update && apt-get install --no-install-recommends -y \
    curl jq tzdata coreutils bash git build-essential ca-certificates \
    stress-ng \
    zlib1g-dev libffi-dev libssl-dev libbz2-dev libncursesw5-dev libgdbm-dev liblzma-dev libsqlite3-dev tk-dev uuid-dev libreadline-dev \
    && apt-get clean && rm -rf /var/lib/apt/lists/*

# install and set up pyenv

terabyte

#!/usr/bin/env bash

set -ex

SSI_DIR="$HOME/projects/ssi"

"$HOME/bin/bat-status-changed.sh"

source "$SSI_DIR/.ve/bin/activate"

terabyte

#!/usr/bin/env python

import io
import json
import random
import sh
import time
import smtplib, ssl

# these are servers that are both close by and in my testing were able to

terabyte

// Copyright (c) 2019 Cloudera, Inc. All rights reserved.

/**
 * Uploads a new GBN
 */

import com.cloudera.dsl.Common

freeStyleJob('PublishGenericGBN') {