api-version-discovery-proposal.rst

Standardized API Version and Capability Discovery for OpenStack

Preface

We are moving into an era of OpenStack where heterogeneous clouds may include services based on differing versions of APIs and with unpredictable extensions and drivers providing differing capabilities for similar service types. Thus it is imperative that we put aside our current habit of assuming or forcibly inserting API versions and capabilities and instead move to a discovery-based mechanism.

"Extensions" vs. "Capabilities"

There has been a fair bit of debate over the terminology of "extensions" or "capabilties". After a significant back-and-forth, there are three defining arguments in favor of capabilities over extensions when discussing this subject:

1. Capability discovery can extend beyond simply enabled/disabled features; it can include information such as maximum capacities, and other more nuanced information.
2. The term "capabilities" separates the meaning of this discovery mechanism from the in-code interpretation of "what is an extension?" which is a very good thing given the current inconsistency with which individual services define the meaning and nature of an extension. By contrast, a "capability" is any piece of funcationality which may or may not be present/enabled and therefore must be discoverable.
3. The /extensions route is alrady in use in some projects and redefining its meaning would be disruptive and backwards incompatible.

How do we get there?

Consensus seems to have formed around a three-part notion:

1. APIs must support:

   • GET / for available version discovery (return code 300),
   • GET /<version> for version availability verification (return code 200),
   • GET /<version>/capabilities for capability discovery (return code 200).

   The return values for these routes needs to be standardized across services (both return codes and some basic level of format). The response body can itself be versioned to provide future-proofing as long as we agree that the top level of the response body will always be a JSON/XML structure which can be parsed into a dictionary and that that top level will contain a key named "version". All subsequent processing can then be handled by a properly-versioned parsing mechanism for compatiblity.

2. The eventual goal is for the Identity Service's service catalog to be configured with endpoints which reflect the root of the service endpoint, and not contain version or project information. A migration path has yet to be determined, but will be considered a requirement for implementation of this proposal.
3. Clients should be able to construct the appropriate endpoints for the service they are designed to talk to. At it's simplest this may not require any work at all for services which don't put project/tenant IDs into their URLs since the links can be found directly in the response for GET /. However in cases where the URL is not so simple the burden lies on the client to do the additional construction.

Ideally code for all of this would live in Oslo so it can be treated as a common reusable library. There is some prior art for this under review here: https://review.openstack.org/#/c/27729/

Diving deeper into point three, we find a few requisite functions:

• A client needs to be able to query the root URL and discover available versions.
• A client needs to be able to choose an appropriate version; default behavior would be to choose the newest version the API and client have in common, with an override mechanism provided to use a specified version (passed in when the client is instantiated).
• A client needs to be able to take versioned endpoint information (likely from the discovery process) and construct a valid service endpoint URL for its use. This may include appending project IDs, etc. to the URL.
• A client needs to be able to accept an "override" endpoint which may contain version information and bypass the discovery mechanism.

Pseudo-code Examples

Basic Auth

Let's take a generic Python OpenStack client as an example (this should look vaguely familiar to most people):

# Accept an unversioned endpoint for auth.
client.Client(auth_url="http://example.com:5000/", user=foo, ...)

Under the hood the client would then query that URL, and discover that the service supports both the v2.0 and v3 API versions at their respective URLs. The client supports v3, so it sets it's internal version to v3 and uses its v3 code paths to do the authentication.

After authenticating it receives back a service catalog with properly unversioned endpoints, and it internally queries those endpoints to determine the appropriate versions just like above for the authentication.

Auth With a Specified Version

Instead of using an unversioned auth URL, what if we wanted to force a v2.0 authentication process:

# Provide a versioned endpoint and manually specify the version
client.Client(auth_url="http://example.com:5000/v2.0", version=2.0, user=foo, ...)

There are three things to note here:

1. Versions should always be numeric, that way they can be compared using standard greater than and less then operators.
2. Despite the fact that we could infer the version from the URL we should not do that. It leads to guesswork and can be prone to error.
3. However, if a user decides to specify only a version number but not a versioned endpoint, the client logic can attempt to determine/infer the correct endpoint.

Skipping Auth and Providing an Override Endpoint

One of the useful features the clients currently provide is the ability to avoid having to auth every request by providing an endpoint and token. In the new scheme that would look very similar, but would have two forms: one versioned and one unversioned:

# Unversioned, client does internal version discovery querying
client.Client(endpoint="http://example.com:8774", token=foo)

# Versioned, client uses specified version
client.Client(endpoint="http://example.com:8774/v2.0", version=2.0, token=foo)

Pretty simple.

Inside the Clients

Let's make sure we cover all the cases:

1. In any case where version is not specified, use the discovery mechanism on the service endpoint. Select the most recent compatible version and construct a vesioned endpoint.
2. If version is specified, query the endpoint to ensure compatibility between the specified endpoint and the specified version.
3. If there is a mismatch between the specified version and the endpoint, or the endpoints supported versions and the client's supported versions then fail gracefully.

Stable vs. Unstable Versions

In the event that unstable versions are specified in the discovery mechanism, the client should select the most recent stable version unless specifically instructed to use the unstable version (e.g. an unstable=True flag or the like). The response from the GET / call would need to have a standardized way of indicating stable vs. unstable versions.

Next Steps

1. Define a common "discovery v1" response format for GET / and GET /<version>/capabilities so that we can get started.
2. Add version discovery/endpoint construction code to all clients. This is likely something we want to write once in Oslo and consume in all the clients.
3. Add GET version discovery support to all service APIs.
4. Drop versioned endpoints from DevStack's service catalog and any other example catalogs as services gain support for discovery.

Currently version discovery (if supported in the clients) would work seamlessly for Keystone, Nova and Glance, so those endpoints can be unversioned in the catalog as soon as client support is present. Other services would need to play catch-up.