specifications for prepare-deployments tool

0-prepare-deployments-specification.md

WARNING: The MEGA team has made some changes to the behaviours and requirements of this tool, we should now consider the cf-deployment README to be the official spec.

This tool will generate manifest(s) referencing the appropriate releases and stemcell, and place manifest files in a given directory. It should have informative messaging to STDOUT and STDERR, but it's just for human convenience, it's not as though the output will be redirected to a file to form a deployment manifest, or anything like that.

Interface Specification

Usage and Inputs

./tools/prepare-deployments <aws|bosh-lite|vsphere|openstack> [path_to_config_file]

The config file provides the following values:

• cf: either a path to a directory of cf-release, or the string "integration-latest"
• etcd: either a path to a directory of etcd-release, a path to a release tarball, the string "director-latest", or the string "integration-latest"
• stemcell: either a path to a stemcell tarball, the string "director-latest", or the string "integration-latest"
• stubs: a list of paths to stub files
• deployments-dir: a path to a directory where manifests will be written

All of the above values are technically optional (even the config file itself), they have defaults, but currently no manifests will generate correctly without at least one stub.

Defaults

• --cf: defaults to "integration-latest"
• --etcd: defaults to "integration-latest"
• --stemcell: defaults to "integration-latest"
• --stubs: defaults to an empty list
• --deployments-dir: defaults to ./outputs/manifests

Behaviour Specification

1. Determine release and stemcell names and versions up front, as well as URLs and SHA1s where relevant:
   a. For every given release and stemcell tarball, peek inside the tarball and figure out the version. The url value should be the absolute path of the file, the version should be the one determined by peeking, and we can provide the sha1 as well. The name of the stemcell should also be determined by peeking. Only accept tarball paths that are absolute. b. For every release or stemcell with the "director-latest" value, just put "latest" for the version, with no url or sha1. For stemcells, we will guess the name based on the infrastructure argument and the integration record. c. For every release or stemcell with the "integration-latest" value:
   --> When it's anything other than cf-release, refer to the integration record in cf-deployment for version (as well as name in the case of stemcell) and include url and sha1 parameters as well, which should already exist in the integration record.
   --> When it's cf-release, the integration record will actually have a git commit SHA; we will clone the repo to a temporary location, check out the SHA, run the ./update script (note this is special cf-release knowledge), and then we can treat this case the same as when given a path to a directory (see below).
   d. For every release given a path to a directory, just put "create" as the version. For url, provide the absolute path of the directory, and don't bother with sha1. Only accept directory paths that are absolute.
2. If the --cf argument is "integration-latest", clone a new copy to some temporary directory, check out the correct SHA, and run the ./update script.
3. Generate the manifests using the provided infrastructure, generated version/file/directory/URL stub from the previous step, and any additional passed-in stubs. Write them to the --deployments-dir location. NOTE: For now, this is just one cf manifest, called cf.yml, but I'd like to be able to split out CATS and smoke-tests manifests into separate deployments eventually.

1-deploy-to-bosh-lite-specification.md

This tool will be a convenience wrapper around the above script, optimized for the BOSH-Lite developer workflow.

Interface Specification

Usage and Inputs

./tools/deploy-to-bosh-lite [path_to_config_file]

The config file specifies the following values:

• cf: either a path to a directory of cf-release, or the string "integration-latest"
• etcd: either a path to a directory of etcd-release, a path to a release tarball, the string "director-latest", or the string "integration-latest"
• stemcell: either a path to a stemcell tarball, the string "director-latest", or the string "integration-latest"
• stubs: a list of paths to stub files
• deployments-dir: a path to a directory where manifests will be written

All of the above values are optional, even the config file itself, the defaults should just work.

Defaults

• cf: defaults to "~/workspace/cf-release" (NOTE: this is different than the previous tool)
• --etcd: defaults to "integration-latest"
• --stemcell: defaults to "integration-latest"
• --stubs: defaults to an empty list
• --deployments-dir: defaults to ./outputs/manifests

Behaviour Specification

1. Make sure it is targetting a BOSH Lite director, and fail otherwise.
2. Generate a director-uuid stub in a temp location.
3. Call ./tools/prepare-deployments, with the "bosh-lite" infrastructure, the generated director UUID stub, any other stubs the existing scripts/generate-bosh-lite-dev-manifest in cf-release pulls in, and then any stubs specified in the config file.

2-integration-record.md

More on the "Integration Record" in cf-deployment

The above assumes a format like the following for the stable record:

releases:
  cf:
    sha: xxx
  etcd:
    version: 3
    url: https://bosh.io/....
    sha1: deadbeef
stemcell:
  aws:
    type: ubuntu-trusty-go-agent-whatever
    version: 9001
    url: https://bosh.io/...
    sha1: deadbeee
  openstack:
    type: ubuntu-trusty-go-agent-whatever
    version: 9001
    url: https://bosh.io/...
    sha1: deadbeed
  vsphere:
    type: ubuntu-trusty-go-agent-whatever
    version: 9001
    url: https://bosh.io/...
    sha1: deadbeec
  bosh-lite:
    type: ubuntu-trusty-go-agent-whatever
    version: 9001
    url: https://bosh.io/...
    sha1: deadbeeb

4-additional-releases.md

Additional Releases

We will need to figure out how new releases (not just etcd and cf) make their way into this. As they do, they should have the same options as etcd.