Create a gist now

Instantly share code, notes, and snippets.

@caridy /View.js Secret
Last active Apr 14, 2016

What would you like to do?
Blog Post: Bending express to support synthetic views

Bending Express to Support Synthetic Views

Express[1] is a wonderful piece of software, not because of the quality of the code and documentation--which are very good--but because of its simplicity. That simplicity, however, comes with a cost that forces users to follow certain guidelines; this is the case of views in Express, where a view has to be bound to a filesystem path. When creating complex applications, there are tricks you can use to get to the next level. Today, I want to touch on the view resolution mechanism in Express and how to tweak it to support what I call synthetic views.

Express supports a variety of view engines, and you can use almost any kind of template language by simply looking for the proper engine implementation for Express. Moreover, creating a new engine is a breeze, and it works wonderfully. Here is an example of Express using @ericf's express3-handlebars[2] package, which adds support for Handlebars[3] templates.

var app = require('express')();

app.engine('hbs', require('express3-handlebars')());
app.set('view engine', 'hbs');

app.get('/', function (req, res, next) {
  res.render('foo');
});
app.listen(3000);

In the example above, when requesting http://localhost:3000/, the template ./views/foo.hbs will be located, compiled, and rendered. As you can see, this is fairly simple, and on top of that, Express will take care of many details, including the cache mechanism for each view, error handling, and filesystem path resolution. It just works and performs remarkably well, but note that it is also bound to a filesystem path to locate foo.hbs.

Express's Internal View Mechanism

The very simple mechanism used by Express to resolve the view is based on few basic configurations:

  • view cache - enables view template compilation caching and is enabled in production by default.
  • view engine - is the default engine extension.
  • views - is the view directory path, which defaults to process.cwd() + '/views'.

and, of course, the view engine registration process:

  • app.engine(ext, callback) - registers the given template engine callback as ext.

Based on those three settings and the registration method, Express will be able to locate, cache, and render any view within the root folder if there is a corresponding engine. When calling res.render(name, options), Express creates a new instance of View, which is an internal component. As part of the constructor, the instance tries to locate the template in the filesystem based on the views setting (using the filename extension from name to detect the proper engine to render the template or the fallbacks to the default engine from view engine setting), and then allocates an internal property called path.

Express' Synthetic Views

What if our templates are not in the filesystem and instead need to be accessed from somewhere else, such as a database, through a REST-like API, or even from memory in the form of compiled JavaScript? How can we gain control over the template resolution? In the end, a view instance is just an object with a render method that expects some data to produce a blob in an asynchronous fashion.

By default, Express implements a View constructor, which takes care of the resolution mechanism. In the past, the View component was private, and there was no way to modify it. That has changed in the express@3.2.0, thanks to @tjholowaychuk for merging our pull request which allows you to a) replace the View constructor and b) modify the View constructor shipped with express. This is also possible through the new view configuration. Here is an example of how to replace the View component:

var app = require('express')();

app.set('view', MyNewViewConstructor);

app.get('/', function (req, res, next) {
  res.render('foo');
});
app.listen(3000);

On the other hand, if you just want to tweak the View component shipped with express, you can still do it by modifying the prototype methods. Here is an example of that:

var app = require('express')();

app.set('view').prototype.lookup = function(name) {
  // and here you can do whatever you want to resolve the template by name
  return myInternalResolver(path); // which returns a path!
};

app.get('/', function (req, res, next) {
    res.render('foo');
});
app.listen(3000);

If you decide to change the View component's prototype, you will have to validate your implementation against new versions of Express because the implementation might change in the future. My recommendation at this point is to replace it with your own implementation as it is a fairly simple component with a very specific responsibility.

[View.js] describes how to implement a View component that can work with a DB, a REST-like API, a global memory hash, or a compiled view accessible through require(). It pretty much covers all the options you have today to fetch and compile a template bound to a view instance.

Compiled Templates for Better Performance and Interoperability

At Yahoo!, we have been trying to create building blocks to help blur the line between the server and client, where sharing code and logic is critical. Knowing that compiling views on the client is just no longer an option, we pre-compiled views into JavaScript for the client runtime. Later, we thought, why not use those same pre-compiled templates on the server as well? It just made a lot of sense.

This use case drove us to propose a change in Express, so we can create a custom View to allow templates to be required and allocated in memory during the application's boot process, when the filesystem representation for the templates is no longer needed and a memory allocation is used instead. That single twist improves performance and reduces the forking in the logic to interoperate between runtimes.

Conclusion

The new view setting introduced in express@3.2.0 provides a simple way to replace or customize the internal mechanism to look up and render views in Express. This new feature can allow us to blur the line between the look up and compile processes as well as eliminate the filesystem requirements, and ultimately enables us to use remote templates, remote compilers, a pre-compiler, and even fetch templates from a database or a REST-like API, while still relying on Express to handle the view instances cache and invocation mechanism.

References

  1. Express (Official Website)
  2. express3-handlebars (GitHub)
  3. Handlebars (Official Website)
var libpath = require('path');
/**
* Initialize a new `View` with the given `name`.
*
* @class View
* @param {String} name the name of the view
* @param {Object} options
* @static
* @constructor
* @api private
*/
function View(name, options) {
options = options || {};
this.name = name;
this.path = name; // this is required, otherwise `express` will assume the instance is not valid
this.root = options.root; // same as app.set('views')
}
/**
* Lookup view by the given `name`.
*
* @method lookup
* @param {String} name the view name which is the first argument when calling `res.render()`
* @param {Object} options the `options` passed as the second argument when calling `res.render()`
* @param {Function} callback the `callback(err, fn)` function where fn represents the compiled view.
* @api private
*/
View.prototype.lookup = function (name, options, callback) {
/**
// option #1: getting compiled template from a nodejs module
// - leveraging require(): assumming the templates were precompiled into nodejs modules
// callback(null, require(libpath.join(this.root, name)));
**/
/**
// option #2: getting compiled template from memory allocation
// - leveraging memory: assumming the templates were precompiled into memory
// callback(null, global.compiledTemplates[name]);
**/
/**
// option #3: getting content to be compiled from a REST-like api
// - mongodb style to fetch a view by name, and compiling it with a global
// compiler method
db.view.find({name: name}, function (err, view) {
if (err) {
return callback(err);
}
callback(null, global.compileView(view));
});
**/
/**
// option #4: getting content to be compiled from a DB
// - using http module to make a GET request to a server using the name as the path to request
require('http').request({
host: 'public-api.com',
port: 80,
path: name,
method: 'GET'
}, function (res) {
var buf = '';
res.setEncoding('utf8');
res.on('data', function(str){ buf += str });
res.on('end', function(){
callback(null, global.compileView(buf));
});
}).end();
**/
};
/**
* Render with the given `options` and callback `fn(err, str)`.
*
* @method render
* @param {Object} options the `options` passed as the second argument when calling `res.render()`
* @param {Function} fn the callback function.
* @api private
*/
View.prototype.render = function (options, fn) {
var path = this.path;
this.lookup(path, options, function (err, template) {
if (err) {
return fn(err);
}
if (!template) {
return fn(new Error('Failed to lookup view "' + path + '"'));
}
fn(null, template(options));
});
};
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment