Exploring Ansible Variables Precedence with hash_behavior = merge A series of experiments was perfomed by defining overlapping dicts in the following files. The integer is to verify that the item was loaded regardless of precedence. Files are given relative to the root ansible-project directory with slashes replaced by spaces.

deploy_vars all ( deploy_vars plugin in vars_plugins folder desends into deploy_vars folder)

td:
  b: deploy_vars/all
  c: deploy_vars/all
  d: deploy_vars/all
  e: deploy_vars/all
  f: deploy_vars/all
  2: deploy_vars/all

Discussion on custom vars_plugin precedence

According to the ansible docs
http://docs.ansible.com/playbooks_variables.html#variable-precedence-where-should-i-put-a-variable

The precedence should be

* -e variables always win (yes)
* then comes "most everything else"
* then comes variables defined in inventory (not tested - hosts vars)
* then comes facts discovered about a system (not tested - gather_facts )
* then "role defaults", which are the most "defaulty" and lose in priority to everything. (no)

Summary

Merge works in general with extra_vars.
For variables in the ansible hierarchy
merge only works within two closely tied pairs
1. roles/defaults/main.yml and roles/vars/main.yml
2. vars found in group_vars or loaded by a vars_plugin


Discovered precedence

most
1. extra vars
---------
| 2. roles/x/defaults/main.yml
| 3. roles/x/vars/main.yml
---------
4. site.yml defined in the vars: section of a play
5. site.yml defined in a vars: section upstream of the play
--------
| 6. group_vars
| 7. deploy_vars (folder depth increases precedence)
--------
8. vars/Debian.yml  where Debian.yml is loaded by a vars_files: in site.yml interpolating on ansible_os_family
9. set_fact: in the task where debug: is called
least

experiments

comment out
none

ok: [default] => {

    "item": "", 

    "msg": "td {u'six': u'extra_vars.yml', 'e': 'roles/ubuntu_common/vars/main.yml', 5: 'ubuntu_common/defaults/main.yml', 'g': u'extra_vars.yml', 'f': u'extra_vars.yml', 'i': u'extra_vars.yml', 'h': u'extra_vars.yml', 'k': 'site_task.yml', u'j': u'extra_vars.yml', 7: 'site_task.yml', 4: 'roles/ubuntu_common/vars/main.yml', 'd': 'roles/ubuntu_common/vars/main.yml'}"

}


comment out 
extra_vars.yml

ok: [default] => {

    "item": "", 

    "msg": "td {'e': 'roles/ubuntu_common/vars/main.yml', 5: 'ubuntu_common/defaults/main.yml', 'g': 'roles/ubuntu_common/vars/main.yml', 'f': 'roles/ubuntu_common/vars/main.yml', 'i': 'site_task.yml', 'h': 'roles/ubuntu_common/vars/main.yml', 'k': 'site_task.yml', 'j': 'site_task.yml', 7: 'site_task.yml', 4: 'roles/ubuntu_common/vars/main.yml', 'd': 'roles/ubuntu_common/vars/main.yml'}"

}

comment out extra_vars.yml, defaults/main.yml, vars/main.yml

ok: [default] => {

    "item": "", 

    "msg": "td {'g': 'site_task.yml', 7: 'site_task.yml', 'i': 'site_task.yml', 'h': 'site_task.yml', 'k': 'site_task.yml', 'j': 'site_task.yml'}"

}

comment out extra_vars.yml, defaults/main.yml, vars/main.yml, site.yml (site_task only)

ok: [default] => {

    "item": "", 

    "msg": "td {'g': 'site_upstream.yml', 8: 'site_upstream.yml', 'h': 'site_upstream.yml', 'k': 'site_upstream.yml', 'j': 'site_upstream.yml', 'i': 'site_upstream.yml'}"

}

comment out extra_vars.yml, defaults/main.yml, vars/main.yml, site.yml 

ok: [default] => {

    "item": "", 

    "msg": "td {'a': 'group_vars/all', 1: 'group_vars/all', 'c': 'deploy_vars/all', 'b': 'deploy_vars/all', 'e': 'deploy_vars/all', 'd': 'deploy_vars/all', 'f': 'deploy_vars/all', 'm': 'bridgepoint/vagrant.yml', 'k': 'bridgepoint/vagrant.yml', 11: 'bridgepoint/vagrant.yml', 12: 'bridgepoint.yml', 2: 'deploy_vars/all', 'o': 'bridgepoint/vagrant.yml', 'l': 'bridgepoint/vagrant.yml', 'p': 'bridgepoint.yml', 'n': 'bridgepoint/vagrant.yml'}"

}

comment out extra_vars.yml, defaults/main.yml, vars/main.yml, site.yml 
group_vars and all deploy_vars tree

ok: [default] => {

    "item": "", 

    "msg": "td {'c': 'vars/Debian.yml', 3: 'vars/Debian.yml', 'e': 'vars/Debian.yml', 'd': 'vars/Debian.yml', 'g': 'vars/Debian.yml', 'f': 'vars/Debian.yml'}"

}

comment out extra_vars.yml, defaults/main.yml, vars/main.yml, site.yml 
group_vars and all deploy_vars tree, vars/Debian.yml

ok: [default] => {

    "item": "", 

    "msg": "td {'i': 'task.yml', 9: 'task.yml', 'k': 'task.yml', 'j': 'task.yml', 'm': 'task.yml', 'l': 'task.yml'}"

}




--------------------- NOTE ON DEBUGING ------------

the debug statement expands msg="td {{td}}" for some variables

TASK: [ubuntu_common | debug msg="td {u'six': u'extra_vars.yml', 4: 'roles/ubuntu_common/vars/main.yml', 'd': 'roles/ubuntu_common/vars/main.yml', 'g': u'extra_vars.yml', 7: 'site_task.yml', 'i': u'extra_vars.yml', 'h': u'extra_vars.yml', 'k': 'site_task.yml', u'f': u'extra_vars.yml', 'j': u'extra_vars.yml', 'e': 'roles/ubuntu_common/vars/main.yml'}"] *** 

<192.168.10.2> ESTABLISH CONNECTION FOR USER: vagrant

ok: [default] => {

    "item": "", 

    "msg": "td {u'six': u'extra_vars.yml', 'e': 'roles/ubuntu_common/vars/main.yml', 5: 'ubuntu_common/defaults/main.yml', 'g': u'extra_vars.yml', 'f': u'extra_vars.yml', 'i': u'extra_vars.yml', 'h': u'extra_vars.yml', 'k': 'site_task.yml', u'j': u'extra_vars.yml', 7: 'site_task.yml', 4: 'roles/ubuntu_common/vars/main.yml', 'd': 'roles/ubuntu_common/vars/main.yml'}"

}


But it stops expanding the debug statement in the TASK: line here
comment out extra_vars.yml, defaults/main.yml, vars/main.yml, site.yml (site_task only)


TASK: [ubuntu_common | debug msg="td {{td}}"] ********************************* 

<192.168.10.2> ESTABLISH CONNECTION FOR USER: vagrant

ok: [default] => {

    "item": "", 

    "msg": "td {'g': 'site_upstream.yml', 8: 'site_upstream.yml', 'h': 'site_upstream.yml', 'k': 'site_upstream.yml', 'j': 'site_upstream.yml', 'i': 'site_upstream.yml'}"

}

extra_vars

append the following data structure with commandline arg --extra_vars "@extra_vars.yml"

where extra_vars contains

td:
  f: extra_vars.yml
  g: extra_vars.yml
  h: extra_vars.yml
  i: extra_vars.yml
  j: extra_vars.yml
  6: extra_vars.yml

group_vars all

td:
  a: group_vars/all
  b: group_vars/all
  c: group_vars/all
  d: group_vars/all
  e: group_vars/all
  1: group_vars/all

roles ubuntu_common task.yml

 - set_fact:
    td:
      i: task.yml
      j: task.yml
      k: task.yml
      l: task.yml
      m: task.yml
      9: task.yml

roles ubuntu_common vars main.yml

td:
  d: roles/ubuntu_common/vars/main.yml
  e: roles/ubuntu_common/vars/main.yml
  f: roles/ubuntu_common/vars/main.yml
  g: roles/ubuntu_common/vars/main.yml
  h: roles/ubuntu_common/vars/main.yml
  4: roles/ubuntu_common/vars/main.yml

ubuntu_common defaults main.yml

td:
  e: ubuntu_common/defaults/main.yml
  f: ubuntu_common/defaults/main.yml
  g: ubuntu_common/defaults/main.yml
  h: ubuntu_common/defaults/main.yml
  i: ubuntu_common/defaults/main.yml
  5: ubuntu_common/defaults/main.yml

vars Debian.yml

td:
  c: vars/Debian.yml
  d: vars/Debian.yml
  e: vars/Debian.yml
  f: vars/Debian.yml
  g: vars/Debian.yml
  3: vars/Debian.yml

The output shows that even with hash_behaviour = merge set in .ansible.cfg the overlapping dicts are merged only in rare cases. Most often, the dicts get clobbered by the highest precedence, perhaps merging with another dict at a similar precedence level (eg. roles/x/defaults/main.yml and roles/x/vars/main.yml)

After each run the dominant contributor is determined from debug output, and the dominant source is commented out for the next run.

Ansible has very basic merge hash behavior test here
https://github.com/ansible/ansible/blob/devel/test/integration/roles/test_hash_behavior/tasks/main.yml
and four test_variable_precedence roles here
https://github.com/ansible/ansible/tree/devel/test/integration/roles

But neither use the canonical ansible tree structure with all the possible places for variables.