Skip to content

Instantly share code, notes, and snippets.

@mattt

mattt/rocket.md

Created May 27, 2015 15:49
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save mattt/62b1318a2a8bec0ed3e7 to your computer and use it in GitHub Desktop.
Save mattt/62b1318a2a8bec0ed3e7 to your computer and use it in GitHub Desktop.
Download ZIP
Rocket: a hybrid approach to real-time cloud applications
Raw
rocket.md

This document was originally posted on 8/1/2013.

Rocket is a technique for building real-time functionality on top of REST web services that leverages web standards like Server-Sent Events and JSON Patch. Most importantly, it fits comfortably with how you're already building applications.

A Tale of Two Paradigms

Just as light can act as both a particle and a wave, so information can be thought as both a document and a stream.

Each approach has its particular strengths and weaknesses:

  • Documents are stateless, long-lived and cacheable, but must be polled for changes.
  • Streams are stateful, ad-hoc, and real-time, but has limited semantics for managing information.

It's clear that real-time will be increasingly important as cloud applications become more connected and ubiquitous, but there will still be a need for the current request-response architecture.

Rocket is a proposal for how to bridge this gap.

According to REST conventions, when a client makes an HTTP GET request to a resource endpoint, a list of records is returned. With Rocket, a client can additionally SUBSCRIBE to changes for that resource by requesting an event stream at that endpoint.

</tr>
Document Stream
Request 
GET /resources
Accept: application/json

SUBSCRIBE /resources
Accept: text/event-stream
Response 
HTTP/1.1 200 OK
Content-Type: application/json

{"resources": [...]}


HTTP/1.1 200 OK
Content-Type: text/event-stream

event: patch
data: [
        {
          "op": "add",
          "path": "/resources/123",
          "value": {...}
        }
      ]

Anytime a record is created, updated, or destroyed, an event is sent in real-time over the event stream, encoding the changes as a JSON Patch document.

Create Update Destroy 
{
  "op": "add",
  "path": "/resources/123",
  "value": {...}
}
 
{
  "op": "replace",
  "path": "/resources/123",
  "value": {...}
}
 
{
  "op": "remove",
  "path": "/resources/123"
}

Building on a Solid Foundation

Building robust, scalable software is difficult, but not nearly as tricky as getting developers to agree on things. Standards, Conventions, and Specifications allow us to stop bike-shedding, and get back to solving real problems.

REST

Representational State Transfer, or REST, is a convention for structuring web application interfaces.

Resources are accessed and manipulated using the existing vocabulary and semantics of HTTP:

  • GET /messages reads a list of messages
  • POST /messages creates a new message
  • GET /messages/123 reads a message
  • PUT /messages/123 updates a message
  • DELETE /messages/123 destroys a message

Server-Sent Events

Server-Sent Events are used to push events to clients over a persistent HTTP connection. An event source stream is designated by the text/event-stream MIME type.

Event messages are composed of newline-delimited fields, each of which contains the field name, followed by a colon, followed by the associated value. The following fields are defined in the specification:

  • event: The event's type.
  • id: The event ID.
  • data: The data field for the message. Multiple lines that begin with "data:" will have their lines concatenated with a newline.
  • retry: The reconnection time to use when attempting to send the event. This must be an integer, specifying the reconnection time in milliseconds. If a non-integer value is specified, the field is ignored.

Although originally designed to send messages to browsers using the EventSource DOM element, Server-Sent Events can be generalized for use with non-browser clients as well.

JSON Patch

JSON Patch describes a common format to represent changes in structured data. A JSON Patch response is designated by the application/json-patch+json MIME type.

A patch is comprised by an array of operations. Six operation types are defined in the specification: add, remove, replace, move, copy, and test. Each operation specifies its type (op), a path, and an optional value or from field:

[
  { "op": "add", "path": "/a/b/c", "value": [ "foo", "bar" ] },
  { "op": "remove", "path": "/a/b/c" },
  { "op": "replace", "path": "/a/b/c", "value": 42 },
  { "op": "move", "from": "/a/b/c", "path": "/a/b/d" },
  { "op": "copy", "from": "/a/b/d", "path": "/a/b/e" },
  { "op": "test", "path": "/a/b/c", "value": "foo" }
]

Although the specification is described primarily in terms of applying a set of changes with an HTTP PATCH request made by the client to the server, the same structure could represent a change on the server that is communicated to clients as a means of resource synchronization.

Reference Implementations

Server

Client

  • AFRocketClient provides a unified client interface to Rocket, making it easy to build real-time functionality on top of an existing REST app.

Contributing

Discussion

If you have any strong ideas about how real-time cloud applications should be structured, it'd be great to get your perspective. Likewise, if you have any concerns or questions about Rocket in its concept or implementation, please share your thoughts.

In any case, feel free to start or join a conversation on GitHub Issues. Or if you're looking for more individual attention, send an e-mail to mattt@heroku.com.

Code

Want to get in on the ground floor of Rocket's ecosystem? Try your hand at implementing Rocket on your favorite server or client language / framework. And if you do get something up and running, let us know by submitting a pull request to mention the project on this page.

Thinking Out Loud

We've only scratched the surface of what's possible with real-time cloud applications. What follows are some random thoughts surrounding this space.

  • Using HTTP for both document and stream access means a single, uniform interface that's compatible without any additional configuration on hosted service providers, such as Heroku or AWS.
  • Rocket could also be implemented using web sockets or third-party push providers as its transport layer without any other changes.
  • The use of the non-standard SUBSCRIBE HTTP method is preferred in order to distinguish from potentially incompatible usage of event streams, though there is no reason why a GET would not also be appropriate.
  • It's yet unclear how a traditional MVC architecture should delegate responsibilities for resource change notifications. Because controllers mediate persistent connections to clients, a strong argument can be made for delegating responsibilities for notification entirely to the controller.
  • That said, if JSON Patch were to be re-appropriated as a wire protocol for replication, changes could be streamed directly to clients from the database... which could be awesome.
  • A pedantic concern: should the event stream response Content-Type somehow encode that the data fields are encoded as JSON Patch documents?
  • Of the commonly-documented software design patterns, is there one that encapsulates an object providing a concrete implementation for executing patch commands on a collection?

References

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment