Horizon Workflows API Proposal

workflows.rst

Workflows API

This proposal describes how the Workflows API should be constructed.

Contents:

Design Goals

The goals of the Workflows API are as follows:

1. Create a simple, easy-to-use Python API for defining multi-step workflows.
2. Ensure that calculating whether or not a user can complete a workflow is possible in advance (including scenarios involving "optional" components and differing role requirements).
3. Allow programmatic definition of richly interactive workflows without needing to write any one-off JavaScript code.
4. Make it easy for 3rd-party code to inject themselves into existing workflows.
5. Automatically adjust the "path" through the workflow based on user roles, presence of "optional" components, and capabilities.
6. Provide reasonable support for non-AJAX interactions.
7. Minimize changes required to existing code.

Central Ideas

The central ideas of the workflow proposal are outlined here:

1. A Workflow is the highest-level grouping, composed of Steps. The workflow maintains a set of "context data" between steps which may influence subsequent steps and is used upon completion to "finish" the workflow.
2. Steps are wrappers around Actions and are aware of the workflow context, required data inputs ("dependencies"), data outputs ("contributions") and dynamic updating between steps ("connections").
3. Actions are the smallest unit of a workflow, and represent a single interaction. They define the form controls for the interaction, and the roles required to complete the action.

Components

Actions

An Action represents an atomic logical interaction you can have with the system. This is easier to understand with a conceptual example: in the context of a "launch instance" workflow, actions would include "naming the instance", "selecting an image or volume", "selecting networks", and ultimately "launching the instance".

Because Actions are always interactive, they always provide form controls, and thus inherit from Django's Form class. However, they have some additional intelligence added to them:

• Actions are aware of roles required to complete them.
• Actions have a meta-level concept of "help text" which is meant to be displayed in such a way as to give context to the action regardless of where the action is presented in a site or workflow.
• Actions understand how to handle their inputs and produce outputs, much like SelfHandlingForm does now.

In code, an action might look like this:

class SelectInstanceBaseAction(workflows.Action):
    image_id = forms.ChoiceField(...)
    volume_id = forms.ChoiceField(...)

    class Meta:
        slug = "select_instance_base"
        title = _("Choose Instance Base")
        required_roles = ("Member",)
        help_text = _("Select the image or volume which you wish to "
                      "use as the base for this instance.")

    def update(self, request):
        # Allow run-time modifications of the action.
        volume_service = get_service_from_catalog("volume")
        image_service = get_service_from_catalog("image")
        if not (image_service or volume_service):
            raise CannotComplete(_("This action cannot be completed..."))
        if not volume_service:
            # Hide the volume-related fields
        if not image_service:
            # Hide the image-related fields

    def clean(self):
        data = self.cleaned_data
        if data['image_id'] and data['volume_id']:
            raise ValidationError(_("Please select an image or a volume, "
                                    "not both."))
        return data

    def handle(self, request, data):
        # Carry out relevant API calls, etc.

Actions as defined here can largely serve as a drop-in replacement for SelfHandlingForm in the current Horizon mapping, and can mostly be based on the existing SelfHandlingForm code, making the transition to Actions fairly straight-forward.

Miscellaneous notes:

• Service requirements--or eventually service capability requirements--can also be specified for an action.

Steps

A step is a wrapper around an action which defines it's context in a workflow. It knows about details such as:

• The workflow's context data (data passed from step to step).
• The data which must be present in the context to begin this step (the step's dependencies).
• The keys which will be added to the context data upon completion of the step.
• The connections between this step's fields and changes in the context data (e.g. if that piece of data changes, what needs to be updated in this step).

In code it would look something like this:

class SelectInstanceBase(workflows.Step):
    action = SelectInstanceBaseAction
    depends_on = ("project_id", "user_id")
    contributes = ("base_type",  # e.g. "image" or "volume" or ???
                   "base_id",
                   "device_name",  # Can be None for an image.
                   "delete_on_terminate")
    connections = {"project_id": ("self.update_images", "self.update_volumes")}

    def update_images(self, project_id):
        # Gets triggered when project_id changes,
        # fetches list of available images.
        return images

    def update_volumes(self, project_id):
        # Same as for images. Illustrates binding multiple connections.
        return volumes

    def contribute(self, data, context):
        image_id = data.pop("image_id", None)
        volume_id = data.pop("volume_id", None)
        if image_id:
            context["base_type"] == "image"
        elif volume_id:
            context["base_type"] == "volume"
        context["base_id"] == image_id or volume_id
        context.update(data)
        return context

Miscellaneous notes:

• Steps inherit the slug and title of the action they wrap.
• The contribute method would just return the data appended to the context as-is by default. We can assume that the data has already been validated before the contribute method is called.
• A step should always list all its outputs in the "contributes" attribute even if the value of those keys may be None. You never know when code outside your control may depend on knowing what you consider the "optional" outputs of your step.

Workflows

A Workflow is a collection of Steps. It's interface is very straightforward, but it is responsible for handling some very important tasks such as:

• Handling the injection, removal, and ordering of arbitrary steps.
• Determining if the workflow can be completed by a given user at runtime based on all available information.
• Dispatching connections between steps to ensure that when context data changes all the applicable callback functions are executed.
• Verifying/validating the overall data integrity and subsequently triggering the final method to complete the workflow.

In code, a workflow is little more than this:

class LaunchInstance(workflows.Workflow):
    slug = "launch_instance"
    default_steps = (SelectInstanceBase,
                     ConfigureNetwork,
                     SetAccessControls)

    def validate(self, data):
        # Handles any extra checks to ensure the sum total data is valid
        return True

    def finalize(self, data):
        # Send all the various API calls in the appropriate order
        return new_instance

Essentially, once the user has completed the workflow (whether in sequence or all at once) the data will be validated and then finalized.

Miscellaneous notes:

• Workflows have the ability to provide compile-time validation that the ordering of steps is logically possible. The application should not be allowed to start with a logically inconsistent workflow.
• The default_steps attribute is named that way to remind people that this is only the stock configuration and may be freely altered by external forces.
• A workflow can have any number of steps omitted/removed as long as the required data is "pre-seeded" into the context data.

Hooking in additional steps

Now that we have a basic understanding of how a workflow is constructed, the next question is extensibility.

Connecting to an existing workflow is done at the Step level, as follows:

from horizon.dashboards.nova.instances.workflows import LaunchInstance

class ExtraStep(workflows.Step):
    # Insert standard step attributes here...
    before = "SetAccessControls"  # Optional
    after = "ConfigureNetwork"    # Optional

LaunchInstance.register(ExtraStep)

These extra couple lines defines which workflow you're inserting the step into, and (optionally) specifies a desired position in the workflow. In a system where multiple steps may request the same position, an absolute position in the order cannot be guaranteed. But this allows you to specify a "best effort" position in the workflow if desired. Otherwise the additional step will simply appear at the end of the workflow.

What happens without AJAX

In an environment without AJAX, two things happen:

• The dynamic updating of fields related to one another simply doesn't happen. The user will not be able to dynamically create new items in lists, etc. This is much like the experience currently.
• The steps in the workflow are essentially "flattened". This means that the steps without dependencies will be grouped into a single sequential form, while steps with dependencies on preceding steps will trigger a "next step"-style click-through workflow.