OperatorHub Index

proposal.md

Design Proposal

To design a migration, we'll paint a picture of a plausible final design and work backward from there.

On top of conformance with the rest of the ecosystem, the value-add of bundles and indexes for upstream-community-operators is a reduction in operator manifests managed in the repository. Ideally, these new features will shrink the operator inclusion mechanism from a set of directories to a finite number of config files.

Once fully migrated to the new formats, the general flow for updating the catalog's content might look like:

1. The git repository is forked
2. A bundle image reference is added, updated, or deny-listed in an index config file
3. A PR is opened against the git repository
4. An index is built with the patched config, using the current production index as a base
5. The index content is verified and tested
6. If tests pass, and the PR is approved, it enters a merge queue
7. If the production index image changes before it merges, the PR drops from the queue (GOTO 4)
8. Once merged, build a new production index image

Now that we've outlined a target flow, we can identify the key migration stages:

1. Only the latest operator manifests stored in the git repository are added to the index image
   • index image is being additively built from previous released versions
   • manifests are still managed in the git repository
   • non-latest versions for package/channels can be removed
   • reduces maintainer burden while waiting for operator authors to start publishing bundle images
2. Both existing operator manifests and bundle images are added to the index image
   • manifests are still managed in the git repository
   • bundle images are also added to the index image
   • existing operator manifests are replaced by bundle images as they become available
3. Only bundle images are added to the index image
   • all existing operator manifests have been replaced by their bundle images
   • PRs adding operator manifests are no longer accepted

In order to drive these stages, we need a configuration file to inform an index build of the bundles to add or remove from an index:

from: image://quay.io/operator-framework/upstream-community-operators:latest
add:
# bundle images
- ref: "image://quay.io/coreos/something-bundle:v1"
- ref: "image://quay.io/coreos/something-bundle@sha256:abcdef..."
- ref: "image://quay.io/redhat/something-else-bundle:v5.5"
# old-format operator manifests referenced by a package file
- ref: "file://upstream-community-operators/etcd/etcd.package.yaml"
# old-format operator manifests referenced by package files in a directory (recursive)
- ref: "file://upstream-community-operators/"
remove:
# remove by bundle image reference
- ref: "image://quay.io/redhat/something-else-bundle:v5.4"
# remove by name
- ref: "name://couchdb-operator.v0.2.0"

Note: The ref field is present to allow for the addition of more fields if necessary

A new -f option will be added to the opm index add subcommand which specifies the path of an index config file. The command will parse the given file and idempotently:

• add bundles referenced by the included directive to the index specified by from
• remove bundles referenced by the excluded directive from the index specified by from

The command will interpret references under the add directive differently based on the given URI-like prefix. These are the supported prefixes and associated behaviors:

• image:// denotes a bundle image to pull
• file:// denotes a file or directory containing bundles to add
  • if the reference is to a file, it is interpreted as a package file and the bundles it references are searched for and added in or below its directory
  • if the reference is to a directory, it is recursively descended and all bundles referenced by package files found therein are added

Similarly, the command will interpret the remove directive using its own set of prefixes:

• image:// denotes a bundle image to remove
• name:// denotes the name of a bundle version to remove

The command will also be capable of sourcing from indexes in varying formats using prefixes as well:

• image:// denotes an index image
• file:// denotes a index database file path