Skip to content

Instantly share code, notes, and snippets.

Last active April 12, 2022 12:54
  • Star 6 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save mhitza/5f3be54ac59d7f0652bd9b803b79f70c to your computer and use it in GitHub Desktop.
Faster Ansible playbook iteration with tags and Vagrant snapshots

In the last few months, I had to write multiple Ansible playbooks, to the point that the slow write/test cycle became a major annoyance. What seemed to work well for me was a mix between Ansible tags and Vagrant snapshots. I would be happy to hear what workflow others employ, that specifically minimizes the time they spend testing.



Vagrant.configure("2") do |config|
  # As a Fedora user I tend to use CentOS for my VMs, as the knowledge accrued in one
  # system translates to knowledge in RedHat/CentOS. Ideally I would use Fedora server all
  # the time. But I don't want to impose my preference, for a less mainstream server
  # distribution, onto my clients.
  # Just using CentOS is enough, as the package manager, default configuration paths and
  # SELinux is a familiar environment. = "centos/8"

  # While this section isn't strictly required, as we will be running Ansible playbook manually,
  # I like to include it as it simplifies sharing with others.
  # While on the topic of sharing, if you're creating a development environment that uses Vagrant
  # and Ansible, I highly recommend ansible_local[1]. That way developers don't need to have Ansible
  # installed locally to bootstrap the box.
  config.vm.provision "ansible" do |ansible|
    ansible.playbook           = "playbook.yml"
    ansible.compatibility_mode = "2.0"
    ansible.inventory_path     = "inventory.ini"
    ansible.limit              = "vagrant"


# Instead of a static ini file it's worth considering writing a shell script[2]. As with multiple
# vagrant machines up at the same time the ansible_ssh_port will differ. Also worth considering
# is that with certain VMs the ansible_private_key_file might be located someplace else.
# The reliable source for ssh configuration parameters can be extracted from the output of
# vagrant ssh-config
# Worth noting that there's also the convenience of setting StrictHostKeyChecking=no inside the
# inventory file. If you're not familiar, ssh creates a fingerprint for each host:port it connects
# to (stored in ~/.ssh/known_hosts), and that information differs between vagrant VMs.
# With this flag we bypass that check, and avoid future connection issues.
[vagrant] ansible_ssh_port=2222 ansible_user=vagrant ansible_private_key_file=".vagrant/machines/default/virtualbox/private_key" ansible_ssh_common_args='-o StrictHostKeyChecking=no'


- hosts: all
  become: yes
    # I use this step to ensure every package I depend on or use is installed inside the VM.
    # Even if some things work out of the box with the Vagrant VM, cloud versions of the same
    # distributions will have a smaller subset of packages installed. This way I cover all bases
    - include_tasks: tasks/setup-system.yml
      tags: ['system']

    # I use the cleanup step to remove any temporary files/directories/utilities that are
    # required during configuration but not on the running system. Most of the time this set of
    # tasks is omitted, but if I'm generating a custom cloud image via Packer this set of tasks
    # are more likely to be present
    - include_tasks: tasks/cleanup-system.yml
      tags: ['system']

- hosts: vagrant
  # Most of the time I use this section to install tools to help me debug configuration issues.
  # vim, netstat, selinux related command line utilities, etc


Since I'm using CentOS most of the time, the first few tasks inside tasks/setup-system.yml will be for enabling additional repositories like EPEL and Remi. Then I spin up the VM and create the first snapshot.

$ vagrant up
$ vagrant snapshot save default system
==> default: Snapshotting the machine as 'system'...
==> default: Snapshot saved! You can restore the snapshot at any time by
==> default: using `vagrant snapshot restore`. You can delete it using
==> default: `vagrant snapshot delete`.

At this point I can start hacking on the configuration tasks following the same approach presented for system setup and cleanup.

What's important to note here, and non-intuitive for me, is that you want to use the --skip-tags Ansible flag, instead of --tags. Given the example playbook if we were to run ansible-playbook -i inventory --tags 'system' playbook.yml none of the tasks defined in tasks/setup-system.yml would run.

$ ansible-playbook -i inventory --tags 'system' playbook.yml

PLAY [all] **************************************************************************************

TASK [Gathering Facts] **************************************************************************
ok: []

TASK [include_tasks] ****************************************************************************
included: /home/user/folder/tasks/setup-system.yml for

PLAY RECAP ************************************************************************************** : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

The reason is that with the --tags flag Ansible will run only the tasks explicitly marked with the specific tags, even if a top level include_tasks/include_role is marked with the tag, all the tasks within that file need to be also marked with the tag.

In our example then, we'll run the playbook with the --skip-tags argument. A bit less convenient but it works.

$ ansible-playbook -i inventory.ini --skip-tags 'all,the,other,tags' playbook.yml 

PLAY [all] **************************************************************************************

TASK [Gathering Facts] **************************************************************************
ok: []

TASK [include_tasks] ****************************************************************************
included: /home/user/folder/tasks/setup-system.yml for

TASK [System - install EPEL repository] *********************************************************

TASK [geerlingguy.repo-epel : Check if EPEL repo is already configured.] ************************
ok: []

TASK [geerlingguy.repo-epel : Install EPEL repo.] ***********************************************
skipping: []

TASK [geerlingguy.repo-epel : Import EPEL GPG key.] *********************************************
skipping: []

Once I'm pleased with the setup for one component of the system (webserver, database, etc), I'll restore my "system" snapshot, run Ansible only with the new section (with the appropriate --skip-tags) preferably twice to ensure nothing gets changed between consecutive runs.

$ vagrant snapshot restore default system --no-provision

If all seems to work well, I then might create a new snapshot that includes the newly configured component and reiterate on the process.

Sometimes I include tags inside my tasks/*.yml files as well. For example, I might have a step that interacts with an API that has rate limits, or package manager installs that add undesired latency to my flow. For those scenarios I use a generic 'skip' tag.

On reddit jnvilo reminded me that you can disable the Gathering Facts step when you're running an Ansible playbook. While it can be set explicitly for each host group within the playbook, I'd rather set it up using an environmental variables. I've kep my original examples the same but I've integrated ANSIBLE_GATHERING=0 ansible-playbook ... in my workflow.



Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment