ansible-acd-redirect-check

.gitignore

*-modules_removed
*.files
*.tar.gz
ansible*/
__pycache__

README.md

ansible-acd-redirect-check

This repo is useless to most people.

It does the following things:

• Compares ansible-2.10 files to ansible-2.9 files to find removed plugins/modules and generates a diff with the results
• Looks for removed_module() skeletons from 2.9 since they were removed in 2.10 and we don't want to count them.
• Checks all the entries of the diff to ensure they live in the ansible_builtin_runtime.yml file in ansible-base
• Iterates over all ansible_builtin_runtime.yml entries and prints the unloadable ones.

Setup

1. Download from pypi: ansible-2.10aN, ansible-2.9.z. Extract them to the same directory as this script so that you have a directory for each of them in the same directory as this README.md.

2. Clone ansible-base with: git clone --depth 1 git://github.com/ansible/ansible ansible-base

3. At this point you should have ansible-2.10a3 (for example), ansible-2.9.10 (for example) and ansible-base directories.

4. Run ./prep.sh to generate file lists and the diff.

5. Run:

export ANSIBLE_COLLECTIONS_PATH=$(echo `pwd`/ansible-2.10*/)

6. Run:

python ansible_builtin_runtime_checker.py ./ansible-base/lib/ansible/config/ansible_builtin_runtime.yml

ansible_builtin_runtime_checker.py

#!/usr/bin/env python

import yaml
import sys

from ansible import constants as C
from ansible.errors import AnsiblePluginRemovedError
from ansible.plugins.loader import *
from ansible.utils.display import Display

C.DEPRECATION_WARNINGS = False

display = Display()

if len(sys.argv) != 2:
    print('Pass a ansible_builtin_runtime.yml file path')
    sys.exit(1)

with open('removals-210.files', 'r') as f:
    removals = f.read().splitlines()

with open('29-modules_removed', 'r') as f:
    module_removals = [l.lstrip('_') for l in f.read().splitlines()]

with open(sys.argv[1], 'r') as abr:
    yaml = yaml.safe_load(abr)

files_removed = {}

for plugin_type in yaml['plugin_routing'].keys():
    # skip these
    if plugin_type in ['module_utils', 'test']:
        continue
    files_removed[plugin_type] = []

for path in removals:
    # Some cleanup
    if path.endswith('/__init__.py'):
        continue

    # First, figure out what kind of plugin it is
    path = path.replace('-lib/ansible/', '')
    # We only really care about module vs plugin here
    # And then if it's a plugin we care about what kind
    if path.startswith('plugins/'):
        sp = path.split('/')
        plugin_type = sp[1]
        plugin_name = sp[2]
        plugin_name = plugin_name.replace('.py', '')
        files_removed[plugin_type].append(plugin_name)
    elif path.startswith('modules/'):
        # For modules, we ONLY care about the name, they all get grouped
        # together.
        filename = path.split('/')[-1]
        filename = filename.replace('.py', '')
        filename = filename.lstrip('_')

        # See if the module was legitimately removed
        if filename in module_removals:
            continue
        files_removed['modules'].append(filename)

print()
print('-' * 72)
print('- Now-ACD plugin listed test')
print('-' * 72)
print()

for plugin_type, plugins in files_removed.items():
    if plugin_type not in yaml['plugin_routing']:
        print('{0} plugin type not found!'.format(plugin_type))
        continue

    for plugin in plugins:
        if (
                plugin not in yaml['plugin_routing'][plugin_type] and
                plugin.replace('.ps1', '') not in yaml['plugin_routing'][plugin_type]
        ):
            display.display('{0}.{1} not found!'.format(plugin_type, plugin), 'red')

print()
print('-' * 72)
print('- Plugin load test')
print('-' * 72)
print()

for plugin_type, plugins in yaml['plugin_routing'].items():
    if plugin_type in ['module_utils', 'test', 'doc_fragments', 'filter', ]:
        continue

    if plugin_type == 'modules':
        loader = globals()['module_loader']
    else:
        loader = globals()[plugin_type + '_loader']

    for plugin in plugins:
        try:
            if not loader.find_plugin(plugin):
                display.display('{0}.{1} not loadable!'.format(plugin_type, plugin), 'blue')
        except Exception as e:
            display.display(str(e), 'red')

prep.sh

#!/usr/bin/env bash
set -e

pushd ansible-2.9*/
  filename29="../$(basename `pwd`).files"
  echo "Writing $filename29"
  find lib/ansible -type f \
      | sed 's|modules.*/\(.\+\)|modules/\1|' \
      | sort \
      > "$filename29"

  removed29="../29-modules_removed"
  echo "Writing $removed29 for removed modules"
  grep 'removed_module(' lib/ansible/modules/ -Rl \
      | sed 's|modules.*/\(.\+\)|modules/\1|' \
      | sed 's|\.py||' \
      | sort \
      | xargs -L 1 basename \
      > "$removed29"
popd

pushd ansible-2.10*/
  filenameACD="../acd.files"
  echo "Writing $filenameACD"
  find ansible_collections -type f | sort > "$filenameACD"
popd

pushd ansible-base*/
  filenameBase="../$(basename `pwd`).files"
  echo "Writing $filenameBase"
  find lib/ansible -type f | sort > "$filenameBase"
popd

echo "Generating list of removals"
diff -ruN "$(basename "$filename29")" "$(basename "$filenameBase")" \
    | grep '^\-lib' \
    | grep -v __init__.py \
    > removals-210.files