Converts from cf job spec properties format to something usable with a normal manifest. Horrendous.

convert_cf_spec.py

import sys
import yaml
import re
import textwrap

data = {}
for k, v in yaml.load(sys.stdin.read())['properties'].items():
    prev = data
    bits = k.split('.')
    for name in bits[:-1]:
        if name not in prev:
            prev[name] = {}
        prev = prev[name]
    prev[bits[-1]] = v

def format(key, items, indent):
    if ('default' in items) or (len(items) == 1 and 'description' in items):
        if items.get('description'):
            description = '\n'.join(textwrap.wrap(items['description'], 80))
            print re.sub('^', '  ' * indent + '# ', description, flags=re.MULTILINE)
        print "%s%s: %s" % ('  ' * indent, key, items.get('default', '~'))
        return

    print "%s%s:" % ('  ' * indent, key)

    for k, v in items.items():
        if isinstance(v, dict):
            format(k, v, indent+1)

format("properties", data, 0)

Example output: https://gist.github.com/Jonty/22efe3658bdf85f261e3

To generate the input from a CF release:

git clone github.com/cloudfoundry/cf-release
cd cf-release
git checkout v225 # Or whatever. This is the build we're currently using for future development.
git submodule update --init --recursive
find . -name spec -type file | xargs spruce merge --prune dependencies --prune packages --prune files --prune excluded_files --prune description --prune name --prune templates

The spruce merge all is a bit dangerous as many times jobs have the same property (e.g. password) and they get overridden. It would be good to write a test to compare if the same job properties (properties with same names between multiple jobs) share same default value or not.