"Occurrences in this domain are beyond the reach of exact prediction because of the variety of factors in operation, not because of any lack of order in nature." - Albert Einstein
Error handling in an asynchronous language works in a unique way and presents many challenges, some unexpected. This tutorial reviews the various error handling patterns available in node, and explains how domains can be used to improve it.
There are four main error handling patterns in node:
- Error return value
- Error throwing
- Error callback
- Error emitting
Before continuing, take a moment to read Guillermo Rauch's A String is not an Error. It is a must read for any error handling discussion.
Returning an error value as the function output is the most basic and limited (it is unusual to find this pattern in node and is included here for the sake of completion). Function performs some activity and return an error if something went wrong.
var validateObject = function (obj) {
if (typeof obj !== 'object') {
return new Error('Invalid object');
}
};
var error = validateObject('123');
if (error) {
console.log('Returned: ' + error.message);
}If the function needs to return a value on valid input and error on invalid, we can check the return value type before using it:
var keysCount = function (obj) {
if (typeof obj !== 'object') {
return new Error('Invalid object');
}
return Object.keys(obj).length;
};
var count = keysCount('123');
if (count instanceof Error) {
console.log('Returned: ' + count.message);
}This pattern is not the most useful for anything beyond simple utilities because it doesn't work asynchronously. We can't use return values when the function has to wait for some event to come back with the results (e.g. database lookup, network activity).
The good news is that this error pattern does not present a error handling problem because it is only available in synchronous operations and always work as expected. If you ignore the error, the application will keep on going without crashing.
Throwing errors is a well-establish pattern, in which a function does its thing and if an error situation arises, it simply bails out throwing an error. It is the responsibility of the caller to catch that error and handle it. Throwing errors is a violent act as it forcefully disrupts the application flow and can leave you in an unstable state.
var validateObject = function (obj) {
if (typeof obj !== 'object') {
throw new Error('Invalid object');
}
};
try {
validateObject('123');
}
catch (err) {
console.log('Thrown: ' + err.message);
}This pattern presents a few challenges. First, it makes it far too easy to ignore errors because it requires extra work to catch them (and the code usually works without it). Second, you need to know when a function may throw an error. If it is not documented, you will have to read the source code (or more likely, find out about it in production when the application crashes).
It can also lead to some unexpected results when working with callbacks:
var parse = function (string, callback) {
try {
callback(null, JSON.parse(string));
}
catch (err) {
console.log('Caught in parse: ' + err.message);
callback(err);
}
};
var display = function (err, obj) {
if (err || !Object.keys(obj).length) {
throw new Error('Empty object');
}
console.log(obj);
};
try {
parse('{}', display);
}
catch (err) {
console.log('Caught outside: ' + err.message);
}Output:
$ node example.js
Caught in parse: Empty object
Caught outside: Empty objectIn this example, we are calling parse() which uses a callback to return the parsed object. We pass display() as the callback which we know throws on empty objects so we wrap this call with a try...catch. parse() calls JSON.parse() within a try...catch to handle parsing errors. However, since the callback chain is called within the inner try...catch, all errors are caught there.
To fix this, we must move something out of the try...catch. We can move parse() out:
parse('{}', function (err, obj) {
try {
display(err, obj);
}
catch (err) {
console.log('Caught outside: ' + err.message);
}
});Or move the callback in parse() out of the try...catch:
var parse = function (string, callback) {
var obj = null;
try {
obj = JSON.parse(string);
}
catch (err) {
console.log('Caught in parse: ' + err.message);
callback(err);
}
callback(null, obj);
};But mostly, once we add asynchronous functionality into the mix, this pattern falls significantly short:
var validateObject = function (obj) {
if (typeof obj !== 'object') {
throw new Error('Invalid object');
}
};
try {
setTimeout(function () {
validateObject('123');
}, 100);
}
catch (err) {
console.log('Thrown: ' + err.message);
}The try...catch will accomplish nothing. By the time the error is thrown, the call is no longer inside the try...catch scope because the timeout execution happens in another tick.
To fix this, we need domains. Domains provide an asynchronous try...catch which is impossible to implement without getting deep into the node code. There are many tricks you can use, but ultimately, it will break on some edge case. Fixing this with a domain is simple:
var domain = require('domain').create();
domain.on('error', function (err) {
console.log('Thrown: ' + err.message);
});
domain.run(function () {
setTimeout(function () {
validateObject('123');
}, 100);
});The run() part is similar to try and the on('error') part is similar to catch. If we don't want to wrap the entire call chain in one big error handler, we can use the domain.bind() method to hand-pick just the functions we want protected:
setTimeout(domain.bind(function () {
validateObject('123');
}), 100);But this is not the prettiest code so we can wrap this in a nicer utility:
var protect = function (run, onError) {
var domain = require('domain').create();
domain.on('error', onError);
domain.run(run);
};And use it as needed:
protect(function () {
setTimeout(function () {
validateObject('123');
}, 100);
},
function (err) {
console.log('Thrown: ' + err.message);
});Now that we have an asynchronous alternative to try...catch, we can look at one more solution to the parse() callback problem we've seen earlier. We can break the event flow by using process.nextTick() or setImmediate() (each has different properties but will work the same for this purpose):
var parse = function (string, callback) {
var reply = function (err, obj) {
process.nextTick(function () {
callback(err, obj);
});
};
try {
reply(null, JSON.parse(string));
}
catch (err) {
console.log('Caught in parse: ' + err.message);
reply(err);
}
};
var display = function (err, obj) {
if (err || !Object.keys(obj).length) {
throw new Error('Empty object');
}
console.log(obj);
};
protect(function () {
parse('{}', display);
},
function (err) {
console.log('Caught outside: ' + err.message);
});Our protect() utility is useful, but limited. It doesn't play well with more complex nested callbacks, similar to the try...catch limitation shown above. We'll come back to it later.
Returning an error via a callback is the most common error handling pattern in node. This pattern is used in core APIs, in modules, and in application code. When a method is called, a callback function is provided. When the method is done, it calls the callback and includes any error encountered as an argument.
var validateObject = function (obj, callback) {
if (typeof obj !== 'object') {
return callback(new Error('Invalid object'));
}
return callback();
};
validateObject('123', function (err) {
console.log('Callback: ' + err.message);
});If the method has an error and a success response, the callback takes two arguments, one for each:
var keysCount = function (obj, callback) {
if (typeof a !== 'object') {
return callback(new Error('Invalid object'));
}
callback(null, Object.keys(a).length);
};
keysCount('123', function (err, count) {
console.log(err ? 'Error: ' + err.message : 'Count: ' + count);
});As an aside, consider using a consistent argument name for all you callback errors such as error, err, or e, and another name for error state you manage locally within a function. This will greatly improve code readability, but will also avoid some common variable name collision errors.
While there is no requirement for the err argument to appear first, it has become an established standard and should be considered a requirement. It is actually a requirement when using domains to handle the callback error state.
Handling error callbacks can become a tangled mess:
var concat = function (a, b, callback) {
if (typeof a !== 'string' || typeof b !== 'string') {
return callback(new Error('Invalid input'));
}
callback(null, a.concat(b));
};
concat('a', 'b', function (err, string) {
if (!err) {
concat(string, 'c', function (err, string) {
if (!err) {
concat(string, 1, function (err, string) {
if (!err) {
console.log(string);
}
else {
console.log('Error: ' + err.message);
}
});
}
else {
console.log('Error: ' + err.message);
}
});
}
else {
console.log('Error: ' + err.message);
}
});Domains to the rescue! It is not always possible to specify a single handler for all callback errors but when it is, as shown above, the code can be simplified:
var domain = require('domain').create();
domain.on('error', function (err) {
console.log('Error: ' + err.message);
});
concat('a', 'b', domain.intercept(function (string) {
concat(string, 'c', domain.intercept(function (string) {
concat(string, 1, domain.intercept(function (string) {
console.log(string);
}));
}));
}));The domain.intercept() method wraps the provided functions with code that automatically looks for an error in the callback's first argument, and when found, stops execution and emits an 'error' event on the domain. It will also "catch" any error thrown inside the function the same way domain.bind() works (the two share much of their internal logic).
Even emitters are a simple but powerful subscription system. When an event is emitted, any subscriber will receive a callback with the event information. When emitting errors, the errors are broadcast to any interested subscribers and handled within the same process tick, in the order subscribed.
var Events = require('events');
var emitter = new Events.EventEmitter();
var validateObject = function (a) {
if (typeof a !== 'object') {
emitter.emit('error', new Error('Invalid object'));
}
};
emitter.on('error', function (err) {
console.log('Emitted: ' + err.message);
});
validateObject('123');Events can use any string name to emit and subscribe. However, the 'error' event name is special. If you emit an 'error' event and there is no one listening, node will throw an error. So while this code will do nothing when no one is subscribed:
var Events = require('events');
var emitter = new Events.EventEmitter();
emitter.emit('some_error', new Error('Something bad happened'));This code will throw an exception which will crash the application if left uncaught:
var Events = require('events');
var emitter = new Events.EventEmitter();
emitter.emit('error', new Error('Something bad happened'));This is a really important fact because it will have significant impact on your application. If you are using any modules or node core facilities (e.g. HTTP server and client) which emit errors, you have to listen for them or your application will crash.
As you can imagine, this can be both frustrating and easy to forget. It is even trickier if a module is emitting errors in some edge case and fails to document this behavior.
You know what's coming... Domains to the rescue!
var Events = require('events');
var domain = require('domain').create();
domain.on('error', function (err) {
console.log('Emitted: ' + err.message);
});
domain.run(function () {
var emitter = new Events.EventEmitter();
emitter.emit('error', new Error('Something bad happened'));
});As expected, the domain catches the emitted error. While this looks like any of the previous examples, it is actually very different. When an EventEmitter is created inside a domain, the domain automatically subscribes to the 'error' event. Internally, this works because the domain is listening to errors, not because it is catching the error thrown due to lack of listeners.
However, this pattern isn't very useful. We don't want to wrap our entire application in one big domain. Instead, we just want to add the EventEmitter to the domain without wrapping it. This is done using the domain.add() method:
var Events = require('events');
var Domain = require('domain');
var domain = Domain.create();
domain.on('error', function (err) {
console.log('Emitted: ' + err.message);
});
var emitter = new Events.EventEmitter();
domain.add(emitter);
emitter.emit('error', new Error('Something bad happened'));With that in mind and everything we've learned before, let's take a look at the domain example in the node documentation:
// create a top-level domain for the server
var serverDomain = domain.create();
serverDomain.run(function () {
// server is created in the scope of serverDomain
http.createServer(function (req, res) {
// req and res are also created in the scope of serverDomain
// however, we'd prefer to have a separate domain for each request.
// create it first thing, and add req and res to it.
var reqd = domain.create();
reqd.add(req);
reqd.add(res);
reqd.on('error', function (er) {
console.error('Error', er, req.url);
try {
res.writeHead(500);
res.end('Error occurred, sorry.');
res.on('close', function () {
// forcibly shut down any other things added to this domain
reqd.dispose();
});
} catch (er) {
console.error('Error sending 500', er, req.url);
// tried our best. clean up anything remaining.
reqd.dispose();
}
});
}).listen(1337);
});With the exception of the domain.dispose() method which will be explained shortly, this should look familiar. We create a domain for the server and instantiate a server inside. Since the node Server is an EventEmitter, it will be implicitly (i.e. automatically) bound to the domain. Same goes for any oncoming request (and matching response) since they are even emitters too.
However, what we want to accomplish here isn't just to catch errors, but to recover from them gracefully - which means always sending a 500 response to the client. To do that, we need a domain per request so that when an error is emitter, we know which request triggered it and (do our best to) send a reply.
You will also notice a try...catch in there. It is there to catch any errors thrown when trying to send the 500 reply. It is using a try...catch and not another domain because all the methods included throw errors immediately in the same tick (but a domain would work just fine as well).
And last, we call domain.dispose() when we are completely done with the request and response. domain.dispose() is a forceful shutdown of the domain and is needed to ensure the domain was properly exited (we never called reqd.run() as the binding to the domain was made via reqd.add()), that the domain is properly removed from any implicitly or explicitly added event emitter, and that any other cleanup task is performed.
We've looked at functions throwing errors and functions returning errors via a callback. But unfortunately, there are many cases where functions do both. There is an ongoing debate about mixing callbacks and error throwing, and both sides have valid arguments.
Regardless, the result is that you need to account for this possibility, as well as consider it when writing your own module for others to use. The linked debate thread is a worthwhile read and contains many useful examples.
There is one more thing to cover about domains: manually entering and exiting. This is done via the domain.enter() and domain.exit() methods. As a general rule, you would be better off not using these methods directly and when possible use the other technique explained above.
However, there are cases where these methods are indispensable. Consider the following scenario:
When adding a route to a web framework like hapi or express, the develop provides a function which is invoked when the route is requested. The framework might not want to wrap the entire request processing in a single domain which makes it harder to keep track of the exact state the request was in when an error occurred. Instead, only the provided handler function is executed within a domain.
The problem is, when the handler method is called, a callback is provided for the handler to notify the framework when it is done (reply() in hapi and next() in express). If the handler is invoked inside the domain, the callback will be too which wraps the rest of the request handling logic in the same domain.
This is similar to the problem we've seen earlier with try...catch, when calling the callback from inside the try statement. Revisiting our protect() method from earlier, we can change it to use domain.enter() and domain.exit():
var protect = function (enter, exit) {
var domain = require('domain').create();
var isFinished = false;
var finish = function () {
if (!isFinished) {
isFinished = true;
domain.exit();
return exit.apply(null, arguments);
}
};
enter(function (run) {
domain.on('error', function (err) {
domain.dispose();
return finish(err);
});
domain.enter();
run();
},
finish);
};And use it to protect the provided handler() method without the domain protecting the postHandler() method which is part of the framework:
var handler = function (request, callback) {
if (!request) {
throw new Error('Missing request');
}
callback(null, 'Success');
};
var postHandler = function (err, obj) {
if (err) {
throw new Error('Handler error');
}
console.log(obj);
};
protect(function (run, next) {
run(function () {
handler(null, next);
});
}, postHandler);The danger of calling domain.enter() and domain.exit() directly instead of domain.run() is that the responsibility of knowing which is the currently active domain falls on the developer. Failing to exit domains in the right order can have unexpected repercussions. For example:
domain1.enter(); // in domain1
domain2.enter(); // in domain2
domain1.exit(); // outside both domainsNode provides multiple error handling patterns. Use them! This is particularly important when developing public modules for others to use. Be consistent, and document your error handling. If you throw or emit errors, let the developers know and make sure error handling is shown in every code example you provide.
As for domains, they are an exciting and welcome addition to the node API. As of node v0.10, they have been promoted from 'experimental' to 'unstable' which means it is still an evolving API. However, they are ready for prime-time and will only get better if more people use them and provide feedback.