Ansible Role Test Shim Script

ansible-role-test.sh

#!/bin/bash
#
# Ansible role test shim.
#
# Usage: [OPTIONS] ./tests/test.sh
#   - distro: a supported Docker distro version (default = "centos7")
#   - playbook: a playbook in the tests directory (default = "test.yml")
#   - role_dir: the directory where the role exists (default = $PWD)
#   - cleanup: whether to remove the Docker container (default = true)
#   - container_id: the --name to set for the container (default = timestamp)
#   - test_idempotence: whether to test playbook's idempotence (default = true)
#
# If you place a requirements.yml file in tests/requirements.yml, the
# requirements listed inside that file will be installed via Ansible Galaxy
# prior to running tests.
#
# License: MIT

# Exit on any individual command failure.
set -e

# Pretty colors.
red='\033[0;31m'
green='\033[0;32m'
neutral='\033[0m'

timestamp=$(date +%s)

# Allow environment variables to override defaults.
distro=${distro:-"rockylinux8"}
playbook=${playbook:-"test.yml"}
role_dir=${role_dir:-"$PWD"}
cleanup=${cleanup:-"true"}
container_id=${container_id:-$timestamp}
test_idempotence=${test_idempotence:-"true"}

## Set up vars for Docker setup.
# Rocky Linux 9
if [ $distro = 'rockylinux9' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Rocky Linux 8
elif [ $distro = 'rockylinux8' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# CentOS 7
elif [ $distro = 'centos7' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Ubuntu 22.04
elif [ $distro = 'ubuntu2204' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Ubuntu 20.04
elif [ $distro = 'ubuntu2004' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Ubuntu 18.04
elif [ $distro = 'ubuntu1804' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Debian 12
elif [ $distro = 'debian12' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Debian 11
elif [ $distro = 'debian11' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Debian 10
elif [ $distro = 'debian10' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Fedora 38
elif [ $distro = 'fedora38' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Fedora 37
elif [ $distro = 'fedora37' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
# Fedora 36
elif [ $distro = 'fedora36' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --cgroupns=host --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:rw"
fi

# Run the container using the supplied OS.
printf ${green}"Starting Docker container: geerlingguy/docker-$distro-ansible."${neutral}"\n"
docker pull geerlingguy/docker-$distro-ansible:latest
docker run --detach --volume="$role_dir":/etc/ansible/roles/role_under_test:rw --name $container_id $opts geerlingguy/docker-$distro-ansible:latest $init

printf "\n"

# Install requirements if `requirements.yml` is present.
if [ -f "$role_dir/tests/requirements.yml" ]; then
  printf ${green}"Requirements file detected; installing dependencies."${neutral}"\n"
  docker exec --tty $container_id env TERM=xterm ansible-galaxy install -r /etc/ansible/roles/role_under_test/tests/requirements.yml
fi

printf "\n"

# Test Ansible syntax.
printf ${green}"Checking Ansible playbook syntax."${neutral}
docker exec --tty $container_id env TERM=xterm ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook --syntax-check

printf "\n"

# Run Ansible playbook.
printf ${green}"Running command: docker exec $container_id env TERM=xterm ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook"${neutral}
docker exec $container_id env TERM=xterm env ANSIBLE_FORCE_COLOR=1 ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook

if [ "$test_idempotence" = true ]; then
  # Run Ansible playbook again (idempotence test).
  printf ${green}"Running playbook again: idempotence test"${neutral}
  idempotence=$(mktemp)
  docker exec $container_id ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook | tee -a $idempotence
  tail $idempotence \
    | grep -q 'changed=0.*failed=0' \
    && (printf ${green}'Idempotence test: pass'${neutral}"\n") \
    || (printf ${red}'Idempotence test: fail'${neutral}"\n" && exit 1)
fi

# Remove the Docker container (if configured).
if [ "$cleanup" = true ]; then
  printf "Removing Docker container...\n"
  docker rm -f $container_id
fi

I'd like to use this in my project. Do you have a license for it?

@cwardgar - There's technically no license attached, but if this comment serves as a license, I license it under the MIT open source license. I guess I could add it in the comments in the actual gist too.

Added debian9, now that my Debian 9 (Stretch) Docker image with Ansible is public.

I ran into a peculiar issue. The first execution of playbook worked well. While trying to run the impotence test, Ansible did not find the installed dependant roles. The error:

`ERROR! the role 'geerlingguy.mysql' was not found in /etc/ansible/roles/role_under_test/tests/roles:/etc/ansible/roles:/usr/share/ansible/roles:/etc/ansible/roles/role_under_test/tests

The error appears to have been in '/etc/ansible/roles/role_under_test/tests/test.yml': line 13, column 7, but may
be elsewhere in the file depending on the exact syntax problem.

The offending line appears to be:

roles:
- role: geerlingguy.mysql
^ here
`
I made a small change in the test shim: docker exec --tty $container_id env TERM=xterm ansible-galaxy install -r /etc/ansible/roles/role_under_test/tests/requirements.yml -p /etc/ansible/roles

Would you modify your script to incorporate -p /etc/ansible/roles? Here's my fork of the gist: https://gist.github.com/bngsudheer/694a1eec8aa5de5f5fda9fc3d81ba12b

Yesterday I added a new option, role_dir, based on a suggestion by @nickjj, which allows you to place the test.sh script anywhere, then run it on a role somewhere else, e.g.

role_dir=/etc/ansible/roles/myrole ./test.sh

awesome job! no more vagrantfile 👍

My directory structure follows one of the scaffolding examples in the ansible best practices guide --
It looks like this:
roles/
tests/test.sh
tests/test-playbook.yaml

The command role_dir=roles distro=ubuntu1604 playbook=test-playbook.yaml tests/test.sh doesn't seem to work -- It gives the following:
ERROR! the playbook: /etc/ansible/roles/role_under_test/tests/base-playbook-ubuntu.yaml could not be found

Could you maybe point out my usage error? Love the concept being implemented here, and was looking to implement it myself, but if I could leverage this off the shelf it would save me a lot of time.

@lyonssp - That should work... not sure why it's not finding that playbook. Can you compare to one of my roles using the script currently? E.g. https://github.com/geerlingguy/ansible-role-apache/blob/master/.travis.yml#L19

I'd just like to note I've been working on converting this into a Go binary.

I've just finished my first successful test, though the tool isn't quite finished.

I've based my program off a script inspired from this one available here:

• https://gist.github.com/samdoran/c3d392ee697881fa33a1d1a65814a07b

This tool will be made public and I will be updating this comment when it is available, and I'll be noting this in the above link too.

edit: The tool I speak of is now public! - https://github.com/fubarhouse/ansible-role-tester

@geerlingguy

Your usage of set -e is a little dangerous.

If any of your Ansible tests fail, it's going to trigger that set -e and halt the script, but at this point your cleanup routine will never get a chance to run.

This puts you into a situation where you end up with a ton of running Docker containers and you'll eventually run out of system resources until you manually stop them all.

I've forked your script to fix this, but the relevant bits are this:

set -eE

# Remove the Docker container (if configured).
cleanup() {
  if [ "$cleanup" = true ]; then
    printf "Removing Docker container...\n"
    docker rm -f $container_id
  fi
}

trap "cleanup" ERR

# ... continue with the rest of your script

# This ensures it gets ran even if no error occurs.
cleanup

Can you please update script? i'm added double quotes for some variables, added support debian 7, and extra_vars variable. This need for run autogenerated tests.

#!/usr/bin/env bash
#
# Ansible role test shim.
#
# Usage: [OPTIONS] ./tests/test.sh
#   - distro: a supported Docker distro version (default = "centos7")
#   - playbook: a playbook in the tests directory (default = "test.yml")
#   - role_dir: the directory where the role exists (default = $PWD)
#   - cleanup: whether to remove the Docker container (default = true)
#   - container_id: the --name to set for the container (default = timestamp)
#   - test_idempotence: whether to test playbook's idempotence (default = true)
#
# If you place a requirements.yml file in tests/requirements.yml, the
# requirements listed inside that file will be installed via Ansible Galaxy
# prior to running tests.
#
# License: MIT

# Exit on any individual command failure.
set -e

# Pretty colors.
red='\033[0;31m'
green='\033[0;32m'
neutral='\033[0m'

timestamp="$(date +%s)"

# Allow environment variables to override defaults.
distro="${distro:-"centos7"}"
playbook="${playbook:-"test.yml"}"
role_dir="${role_dir:-"$PWD"}"
cleanup="${cleanup:-"true"}"
container_id="${container_id:-"$timestamp"}"
test_idempotence="${test_idempotence:-"true"}"

## Set up vars for Docker setup.
# CentOS 7
if [ $distro = 'centos7' ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro"
# CentOS 6
elif [ $distro = 'centos6' ]; then
  init="/sbin/init"
  opts="--privileged"
# Ubuntu 18.04
elif [ $distro = 'ubuntu1804' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro"
# Ubuntu 16.04
elif [ $distro = 'ubuntu1604' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro"
# Ubuntu 14.04
elif [ $distro = 'ubuntu1404' ]; then
  init="/sbin/init"
  opts="--privileged --volume=/var/lib/docker"
# Ubuntu 12.04
elif [ $distro = 'ubuntu1204' ]; then
  init="/sbin/init"
  opts="--privileged --volume=/var/lib/docker"
# Debian 9
elif [ $distro = 'debian9' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro"
# Debian 8
elif [ $distro = 'debian8' ]; then
  init="/lib/systemd/systemd"
  opts="--privileged --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro"
# Ubuntu 12.04
elif [ $distro = 'debian7' ]; then
  init="/sbin/init"
  opts="--privileged --volume=/var/lib/docker"
# Fedora
elif [ $distro ~= fedora?? ]; then
  init="/usr/lib/systemd/systemd"
  opts="--privileged --volume=/var/lib/docker --volume=/sys/fs/cgroup:/sys/fs/cgroup:ro"
fi

#debian:7
# Run the container using the supplied OS.
printf ${green}"Starting Docker container: geerlingguy/docker-$distro-ansible."${neutral}"\n"
docker pull geerlingguy/docker-$distro-ansible:latest
docker run --detach --volume="$role_dir":/etc/ansible/roles/role_under_test:rw --name $container_id $opts geerlingguy/docker-$distro-ansible:latest $init

printf "\n"

# Install requirements if `requirements.yml` is present.
if [ -f "$role_dir/tests/requirements.yml" ]; then
  printf ${green}"Requirements file detected; installing dependencies."${neutral}"\n"
  docker exec --tty $container_id env TERM=xterm ansible-galaxy install -r /etc/ansible/roles/role_under_test/tests/requirements.yml
fi

printf "\n"

# Test Ansible syntax.
printf ${green}"Checking Ansible playbook syntax."${neutral}"\n"
docker exec --tty $container_id env TERM=xterm ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook ${extra_vars} --syntax-check

printf "\n"

# Run Ansible playbook.
printf ${green}"Running command: docker exec $container_id env TERM=xterm env ANSIBLE_FORCE_COLOR=1 ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook ""$extra_vars"${neutral}"\n"
docker exec $container_id env TERM=xterm env ANSIBLE_FORCE_COLOR=1 ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook ${extra_vars}

if [ "$test_idempotence" = true ]; then
  # Run Ansible playbook again (idempotence test).
  printf ${green}"Running playbook again: idempotence test"${neutral}"\n"
  idempotence=$(mktemp)
  docker exec $container_id ansible-playbook /etc/ansible/roles/role_under_test/tests/$playbook ${extra_vars} | tee -a $idempotence
  tail $idempotence \
    | grep -q 'changed=0.*failed=0' \
    && (printf ${green}'Idempotence test: pass'${neutral}"\n") \
    || (printf ${red}'Idempotence test: fail'${neutral}"\n" && exit 1)
fi

# Remove the Docker container (if configured).
if [ "$cleanup" = true ]; then
  printf "Removing Docker container...\n"
  docker rm -f $container_id
fi

How could I pass some skip tags to this script?

---

oh, extra_vars?

This puts you into a situation where you end up with a ton of running Docker containers and you'll eventually run out of system resources until you manually stop them all.

@nickjj - The context for this script is a frequently-wiped CI environment, so I tend to not worry about the longevity of failed containers. For local environments it can be a little annoying... however long term I will be switching my roles over to use Molecule, so I don't know if I'm planning on updating this gist or not besides to fix breaking bugs.

Just noting here that I've removed support for fedora24 (and added fedora29), as well as ubuntu1204, since those images are now marked as deprecated (since the upstream OSes are no longer maintained).

This gist should probably be changed to use suggestions from https://www.jeffgeerling.com/blog/2018/testing-your-ansible-roles-molecule :)

I've just added fedora30 and debian10 to the script. Note that I still use this script in some places where molecule adds more overhead than I need.

Also added centos8 today.

Note that I still use this script in some places where molecule adds more overhead than I need.

@geerlingguy, could you expand that note? When is Molecule's overhead noticeable?

@sio - Sure! The overhead is less on the performance side (things run just as fast with molecule, in general), but more on the side of some projects not needing any of the extra features molecule provides. For example, sometimes I just want to start a container, run a playbook on it, and kill the container. I don't want the extra weight of having to install molecule (and its dependencies), and also manage a molecule.yml file and extra molecule converge playbook just to test another playbook.

Today I dropped Ubuntu 14.04 and Fedora <30, and added Ubuntu 20.04.

I just dropped Fedora < 35, Ubuntu 16.04, and Debian < 10, also dropped CentOS 8 and 6, and added Rocky Linux 9, Debian 12, and all newer versions of Fedora up to 38.