Skip to content

Instantly share code, notes, and snippets.

@smoser
Last active February 10, 2017 16:44
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save smoser/4e25b5a9e6245692be90098464990d00 to your computer and use it in GitHub Desktop.
run curtin vmtest using curtin installed in a container

Now in curtin/tools

This is now in trunk as tools/curtainer

Running curtin's vmtest with curtin installed in a container

Using vmtest to test a distro installed curtin can be done by setting CURTIN_VMTEST_CURTIN_EXE as described in curtin's doc/topics/integration-testing.rst.

This document getting CURTIN_VMTEST_CURTIN_EXE to use an lxc container. The result is you can easily test curtin at different release versions without re-deploying your host OS or installing those packages there.

Setting up a container

A tool is provided here to easily launch an lxc container and install curtin inside it. That is fairly easily done, but this script just makes it a bit easier.

$ ./setup-curtin-container

Usage: setup-curtin-container [ options ] <image> name

   start a container of image (ubuntu-daily:xenial) and install curtin.

   options:
      --proposed    enable proposed
      --daily       enable daily curtin archive

Example:

$ ./setup-curtin-container --proposed ubuntu-daily:yakkety y-proposed

Pointing vmtest at curtin from innside container

A tool is provided here that wraps curtin pack (which is the only command that vmtest uses) and copies files from outside the container inside the container.

The result is that it is easy to do:

$ ./setup-curtin-container --proposed ubuntu-daily:yakkety y-proposed
$ PATH=/path/to/this/dir:$PATH
$ export CURTIN_VMTEST_CURTIN_EXE='curtin-from-container y-proposed curtin'
$ nosetests tests/vmtests/test_simple.py:XenialTestSimple
#!/bin/bash
VERBOSITY=2
error() { echo "$@" 1>&2; }
rerror() { local r=$?; [ $r -eq 0 ] && r=1; error "$@"; return $r; }
fail() { local r=$?; [ $r -eq 0 ] && r=1; failrc "$r" "$@"; }
failrc() { local r=$1; shift; [ $# -eq 0 ] || error "$@"; exit $r; }
debug() {
local level=${1}; shift;
[ "${level}" -gt "${VERBOSITY}" ] && return
error "${@}"
}
Usage() {
cat <<EOF
Usage: ${0##*/} [ options ] container [command [args]]
run command in container with provided args.
This is most useful as a program provided to vmtest
CURTIN_VMTEST_CURTIN_EXE="${0} container curtin" ./tools/jenkins-runner
if 'command and arg1' is 'curtin pack', then it will
read --add arguments to curtin pack, and copy them into a tmpdir container
and adjust the command and args to reference the copied paths.
also can be called by
CURTIN_PACK_CONTAINER=container ${0##*/} [command [args]]
EOF
}
bad_Usage() { Usage 1>&2; [ $# -eq 0 ] || error "$@"; return 1; }
inside() {
local n="$1" close_in=true
shift
[ "$1" = "-" ] && { close_in=false; shift; }
set -- lxc exec --mode=non-interactive "$n" -- "$@"
debug 1 "$@"
if ${close_in}; then
"$@" </dev/null
else
"$@"
fi
}
send() {
local name="$1" prefix="$2" arg="$3"
local afile="" fpath=""
# take ARCHIVE_PATH:FILE_PATH as 'arg'
# push FILE_PATH into $name/prefix/$archive_path
# return prefix/archive_path
afile=${arg%%:*}
fpath=${arg#*:}
lxc file push --create-dirs "$fpath" "$name/$prefix/$afile"
_RET="$afile:$prefix/$afile"
}
packrun() {
local name="$1" tmpd="" ret=$?
shift
cmd=()
while [ $# -ne 0 ]; do
case "$1" in
--) break;;
-a|--add)
from="$1 $2"
arg="$2"
shift 2 || { rerror "failed shift 2 on $1"; return; }
;;
--add=*) from="$1"; arg=${1#*=}; shift;;
*) cmd[${#cmd[@]}]="$1"; shift; continue;;
esac
if [ -z "$tmpd" ]; then
tmpd=$(inside "$name" mktemp -d) ||
{ rerror "failed to make tmpdir in $name"; return; }
fi
send "$name" "$tmpd" "$arg" ||
{ rerror "failed send $arg"; return; }
debug 1 "changed $from => --add=${_RET}"
cmd[${#cmd[@]}]="--add=${_RET}"
done
cmd=( "${cmd[@]}" "$@" )
inside "$name" "${cmd[@]}"
ret=$?
inside "$name" rm -Rf "$tmpd" || {
rerror "failed removing tmpdir in $name";
return;
}
return $ret
}
[ "$1" = "-h" -o "$1" = "--help" ] && { Usage; exit 0; }
if [ -n "${CURTIN_PACK_CONTAINER}" ]; then
container="${CURTIN_PACK_CONTAINER}"
[ $# -gt 0 ] ||
fail "must give command (CURTIN_PACK_CONTAINER=${container})"
else
[ $# -gt 1 ] ||
fail "must give container and command (or set CURTIN_PACK_CONTAINER)"
container="$1"
shift
fi
out=$(lxc info "$container") ||
fail "failed 'lxc info $container'. container '$container' does not exist?"
if [ "${1##*/}" = "curtin" -a "$2" = "pack" ]; then
packrun "$container" "$@"
else
inside "$container" "$@"
fi
# vi: ts=4 expandtab
#!/bin/bash
VERBOSITY=0
CONTAINER=""
error() { echo "$@" 1>&2; }
fail() { local r=$?; [ $r -eq 0 ] && r=1; failrc "$r" "$@"; }
failrc() { local r=$1; shift; [ $# -eq 0 ] || error "$@"; exit $r; }
Usage() {
cat <<EOF
Usage: ${0##*/} [ options ] <image> name
start a container of image (ubuntu-daily:xenial) and install curtin.
options:
--proposed enable proposed
--daily enable daily curtin archive
EOF
}
bad_Usage() { Usage 1>&2; [ $# -eq 0 ] || error "$@"; return 1; }
cleanup() {
if [ -n "$CONTAINER" ]; then
debug 1 "deleting container $CONTAINER"
lxc delete --force "$CONTAINER"
fi
}
inside() {
local n="$1" close_in=true
shift
[ "$1" = "-" ] && { close_in=false; shift; }
set -- lxc exec --mode=non-interactive "$n" -- "$@"
debug 1 "$close_in: $*"
if ${close_in}; then
"$@" </dev/null
else
"$@"
fi
}
wait_for_ready() {
local n="$1" max="${2:-30}" debug=${3}
inside "$n" - /bin/sh -s $max $debug <<"EOF"
max=$1; debug=${2:-0};
i=0;
while [ ! -e /run/cloud-init/result.json ] && i=$(($i+1)); do
[ $i -ge $max ] && exit 1
[ "$debug" = "0" ] || echo -n .
sleep 1
done
[ "$debug" = "0" ] || echo "[done after $i]"
exit 0
}
EOF
}
debug() {
local level=${1}; shift;
[ "${level}" -gt "${VERBOSITY}" ] && return
error "${@}"
}
main() {
local short_opts="hv"
local long_opts="help,proposed,daily,verbose"
local getopt_out=""
getopt_out=$(getopt --name "${0##*/}" \
--options "${short_opts}" --long "${long_opts}" -- "$@") &&
eval set -- "${getopt_out}" ||
{ bad_Usage; return; }
local cur="" next=""
local proposed=false daily=false src="" name="" maxwait=30
local eatmydata="eatmydata"
while [ $# -ne 0 ]; do
cur="$1"; next="$2";
case "$cur" in
-h|--help) Usage ; exit 0;;
--proposed) proposed=true;;
--daily) daily=true;;
-v|--verbose) VERBOSITY=$((${VERBOSITY}+1));;
--) shift; break;;
esac
shift;
done
[ $# -eq 2 ] || { bad_Usage "expected 2 args, got $#: $*"; return; }
trap cleanup EXIT
src="$1"
name="$2"
lxc launch "$src" "$name" || failrc "failed lxc launch $src $name"
CONTAINER=$name
wait_for_ready "$name" $maxwait $VERBOSITY ||
failrc "$name did not become ready after $maxwait"
if $proposed; then
mirror=$(inside $name awk '$1 == "deb" { print $2; exit(0); }' \
/etc/apt/sources.list) &&
rel=$(inside $name lsb_release -sc) ||
fail "failed to get mirror in $name"
line="deb $mirror $rel-proposed main universe"
debug 1 "enabling proposed: $line"
local fname="/etc/apt/sources.list.d/proposed.list"
inside "$name" sh -c "echo $line > $fname" ||
fail "failed adding proposed to sources.list"
fi
if $daily; then
local daily_ppa="ppa:curtin-dev/daily"
debug 1 "enabling daily: $daily_ppa"
inside "$name" add-apt-repository --yes "${daily_ppa}" ||
fail "failed add-apt-repository for daily."
fi
inside "$name" $eatmydata apt-get -q update ||
fail "failed apt-get update"
inside "$name" $eatmydata \
apt-get install --no-install-recommends -qy curtin ||
fail "failed apt-get install --no-install-recommends -qy curtin"
CONTAINER=""
}
main "$@"
# vi: ts=4 expandtab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment