server-side.md

Thorax Server Side Rendering

WARN: This is preliminary documentation and subject to change.

There are three major concerns that need to be solved to do full server-side rendering of our current code base, server-based routing, HTML rendering and restoring views once recieved on the client.

Server Routing

Server routing is built on top of Hapi's routing stack. At build time all backbone routes are converted to Hapi routes which the Scotsman project registers on server startup. This is done through the module-map.json config file, included in the static resource bundle.

Initially Backbone routing was used directly but this conflicted with other path* cases on the server so individual routes are now registered.

On each route the same handler is used, which delegates to the HTML rendering engine described below.

HTML Rendering

The HTML rendering concept is one of a subset of the $ APIs rather than the full DOM. For cases where there are conflicts or code is not desired on the server for performance reasons conditionals may be done at build time via the Lumbar "server": false flag on the resource or at runtime (to be optimized out via Uglify) via the $serverSide global flag.

Some things that are explicitly not supported are:

• setTimeout / setInterval nextTick is available for code that is explicitly server aware and needs to dump off the stack. A client-side implementation of nextTick is provided as well to provide transparent execution on either stack.
• DOM Events
• Reflow/Layout logic and APIs
• Local Storage and Cookies (TBD)

Implementations of unsupported APIs range from NOP stubs to omission of the API entierly.

In practice most of the server conditionals have been in the framework level code with this setup or involve omitting particular components from the server-side build. Some UI components that are a bit more complex, such as the carousels, will unfortunately require conditonal logic based on where they are being rendered as well as some hueristics to account for data that is missing from one case or the other but the hope is that divergent code will be handled primarily within the framework vs. application logic.

Once the request reaches the rendering handlers the control flow occurs in much the same manner as on a normal webpage. The HTML will be parsed, scripts loaded, Backbone routes, and the router will execute as normal.

The request is finalized when one of two conditions occurs:

1. setView is called with serverRender flag on both options and and view instance falsy. In this situation the response will be emitted and further processing deferred.
2. The global emit is called.

Should multiple emit events occur, the first will win. The second will be logged but the user will not see any of the changed output. This highlights an important caveat of the implementation, once a request is sent it's possible that it's still maintaining resources on the server. Things like AJAX calls will be cancelled but nextTick chains can take up server resources indefinately. This is why setInterval is explicitly disallowed.

On emit the following will be sent back to the user inlined in a single HTML document:

• CSS loaded
• HTML
• Services responses loaded

This is all building on top of the following stack:

• Node
• Hapi
• Cheerio
• Fruit Loops (In house library that implements most of the lifecycle logic above)

It's possible that we could also investigate a lighter weight implementation that utilizes Hapi to route directly to the Backbone routers but there is enough initilization code in the average app that are independent of the routers that this is being treated as a premature optimization. The same goes for attempting to cache the DOM and JS in a particular state, i.e. a pre-router cached javascript context.

To allow for aggressive caching, Javascript Edge Caching may optionally be implemented under this system. When configured (with associated metadata on data requests) this will differentiate between user specific data and public data. User data will not be requested on the server, instead the client will make said request and augment the view once the data is available. This allows the cache timelines to be maximized, reducing event loop overhead, etc.

Regardless of serving user specific data or not, the overal response's TTL should be derived from the data sources hit, selecting the most restrictive cache policy. This combined with Hapi's soon to be related setState cache restriction should help reduce concerns of caches that live far longer than desired.

Restoring Views on Rendered HTML

Once the client recieves the HTML, the intialization process is much the same as the current client-side process. The standard page components are parsed and rendered, Backbone inits and routes. The exception is that at various points in the initialization process we are going to attempt to associate view objects with existing HTML rather than creating new. This allows for us to wire any events to the rendered HTML as well as provide updates when new content needs to be rendered.

This presents a number of problems as it's not always clear as to what elements should be associated with a particular javascript object after transitioning to the client-side javascript environment.

A number of solutions, of varing levels of sanity, have been considered at this point:

• Including the heap information with the response
  • Insane
• Using view cid
  • Breaks when optimizing server content as cid is shared and serial
• Rerendering the entire view after JS was loaded
  • Overwriting normal rendered content
    • UI flashes seen under Chrome Desktop when experimenting with this.
  • Inserting the server content into a NOSCRIPT tag and rendering the client content after everything is recieved.
    • Might make the google bot happy but penalizes the common case as there is a fairly substantial amount of unused data sent on the wire that defers the rendering of the content the client actually uses.
• Hueristically walking the JS heap based on the contents of the template

While none of these are perfect solutions the later solution is the one being actively investigated.

The basic premise is that any data available is utilized to map the views to the elements and should that fail for whatever reason the view and any unattached subviews will be rerendered.

The hueristics currently targeted are:

• Direct instantiation Used for {{view "registry"}} and various {{collection}} view rendering cases.
• Handlebars "path" Using Handlebars id tracking mode, a hybrid of the Handlebars stringParams mode, where paths relative to the view and context are tracked. On restore these are dereferenced. For simple cases this works quite well but has a number of cases where it can break down. Depthed references (../foo) and helpers that change the context and don't implement the approate path changes can all cause this to failover to rerendering.
• Model id references More of an additional data point, but views with an associated model are annotated as such and used to compare with the candidate element, if available.
• View.el selector wiring

For each of these cases there are a variety of sanity checks that will err on the side of rerendering rather than attempting to reuse content of unknown origin.

There are some assumptions made with this implementation. The biggest being that the data generated in one place will be the same as that generated in another. This could be impacted by things like client-side sorting and, when in Edge mode, user specific differences such as AB tests, could cause this to break down. This is the reason for the opt-in behavior when actually sending a view's contents, vs. JSON content only, to the client.