lox

steps:
  - command: echo hello world
    parallelism: 4
    plugins:
      - docker#v1.4.0:
          image: "alpine:latest"

lox

#!/bin/bash
set -euo pipefail

input="$(cat <&0)"

if [[ "${COMPARE_EVAL:-}" == "1" ]] ; then
  printf 'parsing of args with native bash eval:\n\n'
  eval "array=($input)"
  for token in "${array[@]}"; do
    printf '[%q]\n' "$token"

lox

steps:
  - command: sleep 60

lox

Overview

We have an autoscaling group of m5.large instances with 250GB EBS root volumes that we use for running our Buildkite test suites. This group scales up from 0 to 40 during the work day and then down again, so each day we see new instances.

We've been seeing every few days that a few of the instances run very, very slowly. Our test suite either takes 100x it's usual time or times out entirely. On the host machine, a basic command like aws s3api head-object --bucket blah --key blah will take 45 seconds.

All instances are m5.large in us-east-1 running the latest Amazon Linux 2 ami-06631de3819cb42f3

Tests

lox

FROM lgatica/openssh-client:latest

lox

steps:
  - command: test.sh
    plugins:
      docker-compose#v2.4.1:
        run: app

lox

Often inside a docker container one wants to run specific commands on the host. binproxy provides a way to allow specific command invocations inside a docker container that proxies to commands on the host:

commands:
  buildkite-agent:
    parameters:
      - regexp: '^meta-data (set|get)'
    env:
      - BUILDKITE_AGENT_ACCESS_TOKEN

lox

#!/bin/bash

buildkite-agent artifact download test.txt .
cat test.txt

lox

steps:
  - label: "Test local plugin"
    agents:
      queue: "${BUILDKITE_AGENT_META_DATA_QUEUE:-default}"
    plugins:
      "/Users/lachlan/Projects/buildkite/docker-compose-buildkite-plugin": ~

lox

FROM alpine:3.3
RUN echo "CACHE MISS" && sleep 10
RUN apk update && \
     apk add --no-cache openssl && \
     rm -rf /var/cache/apk/*
RUN echo "CACHE MISS" && sleep 10
RUN openssl req -x509 -nodes -newkey rsa:4096 -keyout /key.pem -out /certificate.pem -days 720 -subj "/C=US/ST=State/L=City/O=Department/OU=Company/CN=localhost.localdomain"
RUN echo "CACHE MISS" && sleep 10