TaskOrientedAPIs.md

Task Oriented API's

An immediate challenge when moving from an JSON RPC API to a Hypermedia API is how to deal with complex objects.

With an RPC API, the expectation is closer to a batch processing system then a RESTful one. The object structure, including nested and complex types, is pre-communicated through documentation, and is passed as a single job, or entity to be processed in one fell swoop. This has many advantages when doing batch jobs, between servers.

For our example, we'll evaluate an invoicing system which creates new invoices with an arbitrary number of line-items. Each of these line items can be of differing, specific types. Since we <3 FQDN, we'll use invoices.example.com as our domain. To start, we'll examine a familiar JSON RPC example.

https://invoices.example.com/invoices

a GET to this location, we can expect to find a collection:

[{
	"id": "239470-ua190u",
	"clientId": "ac8689c9-d3c0-4e46-9a42-f4944b8a340a",
	"date": "2016-02-04T20:33:13.289Z",
	"total": 20293.30,
	"currency": "USD",
	"status": "unpaid",
	"lineItems":[{
		"type": "setup-fee",
		"id": "21a",
		"description": "One time account activation fee",
		"total": 10000.00
	}, {
		"type": "subscription",
		"id": "2310b",
		"description": "1yr Gold Member",
		"total": 10293.30
	}, {
		"type": "subscription",
		"id": "12",
		"description": "1yr Consultancy subscription",
		"total": 0.00
	}]
}, {
	...
}]

New invoices can created and added to this collection by POSTing an invoice, with AT LEAST 1 valid line item, defined by our documentation. The only field omitted here is the id, this is provided in the response from the server.

A given invoice can be updated by putting back a modified object to the route with the invoice's id templated into it.

https://invoices.example.com/invoices/239470-ua190u

This workflow makes several assumptions:

• the client knows what valid line items are (api calls MUST have been made previous to discover these)
• the client knows which line items go together
• the client knows where to find clientId, and already has one
• the client knows how to change the status to valid states: unpaid, paid, processing etc.

These assumptions put more burden on the client to have more information, but allow it to do more things in less steps. This is again useful for batch processing, and machine-to-machine interaction which require strict contracts and expert clients.

Assume Less

Lets now pretend the client doesn't have the rules for line items: which ones work, which ones are valid combinations, what the clientId is etc. How could an api-client go about creating a new invoice. The answer is to break these assumptions out and engage in a workflow to complete this task.

We'll re-represent this same example with Hypermedia, specificially Siren

Like all good hypermedia clients, we would start at the root https://invoices.example.com/

here we receive the response

{
	"actions": [{
		"name": "create-new-invoice",
		"title": "Create a new Invoice",
		"method": "GET",
		"href": "https://invoices.example.com/create/1"
	}],
	"links": [{
		"rel": ["self"], "href": "https://invoices.example.com/",
	}, {
		"rel": ["https://invoices.example.com/rels#invoices"], "href": "https://invoices.example.com/invoices"
	}]
}

This action basically tells us that creating a new invoice starts at href": "https://invoices.example.com/create/1. Going to there, we are presented with a collections of clients to select, in that an invoice must be attached to a client.

{
	"class": ["client", "collection"],
	"entities": [{
		"class": ["client"],
		"properties": {
			"name": "Techtronics Computer Canada LTD.",
			"phone": {
				"fax": "(306) 242-6823",
				"primary": "(306) 668-0118"
			},
			"address": {
				"number": "1738",
				"street": "Quebec Avenue",
				"unit": "Unit 26",
				"postal": "S7K 1V9"
			}
		},
		"actions": [{
			"name": "select-client",
			"method": "GET",
			"href": "https://invoices.example.com/create/2?state=%7B%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D"
		}],
		"links": [{
			"rel": ["alternate"], "href": "https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a"
		}]
	}, {
		...Many More Clients...
	}],
	"links": [{
		"rel": ["self"], "href": "https://invoices.example.com/create/1"
	}, {
		"rel": ["https://clients.example.com/rels#client"], "href": "https://clients.example.com/clients"
	}]
}

Executing the client action sends us to the next step in the workflow, which the server has determined is by sending the client's href to https://invoices.example.com/create/2. The server now has the information it needs to know which client it should assign the invoice to.

{
	"actions": [{
		"name": "create-invoice",
		"title": "Create invoice for Techtronics",
		"method": "POST",
		"href": "https://invoices.example.com/create/3?state=%7B%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D",
		"fields": [{
			"name": "title", "type": "text", "value": "Techtronics Invoice #29"
		},{
			"name": "description", "type": "text"
		}]
	}],
	"links": [{
		"rel": ["self"], "href": "https://invoices.example.com/create/2?state=%7B%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D"
	}],
}

Fun note: Because the server already knows the client you were creating the invoice for, it can do helpful things like fill in a default title with information the client lacks, like which invoice number this would be fore this client.

POSTing this object might return In our example, we stated (arbitrarily) you cannot create an invoice without at least one valid line item.

	{
		"class": ["invoice"],
		"properties": {
			"name": "Techtronics Invoice #29",
			"description": ""
		},
		"actions": [{
			"name": "add-setup-fee-line-item",
			"title": "Add Setup Fee",
			"method": "POST",
			"href": "https://invoices.example.com/create/4?state=%7B%22title%22:%22Techtronics%20Invoice%20#29%22,%22description%22:%22%22,%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D",
			"fields": [{
				"name": "title", "type": "text", "value": "21a"
			}, {
				"name": "total", "type": "number", "value": 10000.00
			}],
		}, {
			"name": "add-gold-subscription-gold-1yr-line-item",
			"title": "Add 1yr Gold Subscription",
			"method": "POST",
			"href": "https://invoices.example.com/create/4?state=%7B%22title%22:%22Techtronics%20Invoice%20#29%22,%22description%22:%22%22,%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D",
			"fields": [{
				"name": "title", "type": "text", "value": "2310b"
			}, {
				"name": "total", "type": "number", "value": 10293.30
			}],
		}, {
			"name": "add-consultancy-subscription-1yr-line-item",
			"title": "Add 1yr Consultancy Subscription",
			"method": "POST",
			"href": "https://invoices.example.com/create/4?state=%7B%22title%22:%22Techtronics%20Invoice%20#29%22,%22description%22:%22%22,%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D",
			"fields": [{
				"name": "title", "type": "text", "value": "12"
			}, {
				"name": "total", "type": "number", "value": 0.00
			}],
		}, {
			... many more ...
		}],
		"links": [{
			"rel": ["self"], "https://invoices.example.com/create/3?state=%7B%22title%22:%22Techtronics%20Invoice%20#29%22,%22description%22:%22%22,%22client%22:%22https://clients.example.com/clients/ac8689c9-d3c0-4e46-9a42-f4944b8a340a%22%7D"
		}],
	}

Here is where the server being the authority comes back into play:

• all valid line items, their ID's and default values are provided
• the action 'complete-invoice-creation' isn't present, because it isn't a valid action until at least one line item is selected

Assuming we add one of those line items, and we see the 'complete-invoice-creation', we should be good to go.

Observation: Holy crap that was a lot of steps

True! It could be argued that it is a similar number of steps regardless of if you use hypermedia or if you use RPC JSON.

• both require you have acquired the clientId. With RPC JSON, you'd probably called /clients to get the ID's first, this would be a hard-coded call to an endpoint which means it can't change. Contrast this with they Hypermedia workflow: Doesn't matter where the users are, they've been included in the invoice workflow with the correct action for the invoice workflow added to them

• both require you have acquired valid representations (from the server) for line-items. The RPC workflow would probably have called /line-items first to get a list. Contrast this with the Hypermedia workflow: line items are included and have actions present because the action is allowed, as per the server.

• the RPC workflow must have the knowledge of which line-items work together codified in the client logic

• the PRC workflow must have the knowledge of how many line-items are required, codified in the client logic

  • if either of these rules change, the client must be recoded and would be incorrect till they are

The number of steps to create the JSON object we are sending to the server are identical; One workflow is implemented with logic in the client to create a hopefully valid JSON object, the other is implemented on the server.

One of these is more likely to be correct than the other.

The clearest difference is the size of the responses. As has been observed before, Hypermedia directives add bulk to requests. The repetition of serialized state seems super redundant. We have also observed this repeated data being moot when gzip is applied.

A query to our current valence apis for 900 enrollments zips down to 15Kb. The same request decorated with full hypermedia controls, FQDN-esqe links and classes zips down to 16.3Kb.

The same request decorated with full hypermedia controls, FQDN-esqe links and classes zips down to 16.3Kb.
Does that include 'forms' (fields) to manipulate the data? Or just highly compressible URLs that share bloaty parts?

I'm not sure your RPC comparison is fair in the context of batch processing. For example, often the case with RPC it is such that you have an ID and you want to engage some mechanism that creates or alter state - so, in response to something, I want to create an invoice for the client that triggered the action. (eg, an event occurs and then based on that event, which might include a ClientId, I want to create a new Invoice for that client.) -- in this case, the problem is that you are saying I would have to fetch a valid client id. Often I don't. I just create a new Invoice(clientId) and sure, if you adjust the validation steps, it fails -- but the same thing is true for REST interfaces. You adjust the validation steps or introduce a new part of a task, and unless the automation is adaptive and intelligent it fails.

Consider that the 'requires a line item' case. Well, introducing that constraint would break clients that don't expect to select something. Some degree of prior knowledge is still baked in. In your case, you're saying if the # of required line items change.. well, even in the hypermedia case you would have a task that is trying to create an invoice. Does it just repeatedly keep selecting line items until its done? Sure, if a user is performing the action they can read prompts and see things.

Hypermedia, task-oriented intefaces seem excellent for users, and suboptimal for machines.

Does that include 'forms' (fields) to manipulate the data? Or just highly compressible URLs that share bloaty parts?

It's the same request/response as valence, so it has no actions included, as valence doesn't. This was more apples to apples, decorated with class information, having links to self, properties etc, for 900 items, it adds 1.3Kb. Adding more functionality would have to add more size. The point there was that equivalent amounts functionality produce very similar response sizes after gzip is applied, despite hypermedia decoration.

Your points on the RPC comparison for batching are spot on. I tried to call out the case for batching being separate and apart from the workflow example, but I'll incorporate your points to make it clearer. Server-to-server clients are probably going to start with all the data and want to create multiple invoices with all their line items in one call. They do have all the constraints mentioned above for expectations in the client, because they'd have to. They would have to have codified all those rules for line items and would stop working when any of those rules changed. If the rule changed in either case, RPC or Hypermedia, a client without the expectation would break. At that point, any client would have to be updated with the new 'rule' if its making decisions. That all seems like worthwhile things to add in.

Thanks for the feedback!