Promise Inspector Protocol

promise-inspector-protocol.md

One of the features of promises, in the Promises/A+ sense, is that they capture thrown exceptions so they can be handled asynchronously. This has the negative consequence that it is possible for an error to be silently ignored forever. There is a long list of measures that can mitigate this problem, but the real solution is to use a Promise Inspector.

The JavaScript console is not satisfying. The trouble with a rejected promise is that it might not be later connected to a promise chain that would handle recover from or surface the issue. The naïve solution would be to log every exception to the console, but then it becomes difficult to distinguish errors that have been handled programmatically from errors that never will be. Ideally, the promise inspector would show rejected promises until they are handled and then hide them. That way, if a program reaches an obviously stable or idle state and a rejection remains on the console, it is very likely due to a programmer’s error.

There is a less obvious class of problems a promise inspector can surface: the forever pending promise. Consider a promise inspector that shows a pending promise until it is either fulfilled or rejected. If a program reaches an obviously idle or stable state and a pending promise remains in the inspector, it is the harbinger of a problem, and the best place to start inspecting.

There is precedent for promise inspectors. The most noteworthy is Causeway, a promise debugger designed for Mark Miller’s E programming language. Ember Extension has recently introduced a promise inspector. There is also a proof-of-concept for a Chrome Extension that introduces a Promise inspector for the Q promise library.

It would also be useful to look into CapTP and CapNProto for prior art.

Promises come and go over the life of a program. As such, it is not practical to retain the entire history of promises in the memory of either a promise inspector or a promise library. It is however practical for a library to retain an index of all pending promises and all promises that have been rejected but not yet handled. It would also be practical for a promise inspector to retain a circular buffer of a limited size for all other promises for the purpose of tracing.

The scope and life of a promise inspector panel and inspected windows overlap but are disjoint. Ideally, there would be one promise inspector panel for each window under test and their lives would be coordinated, but this is not the case for multiple reasons. Reloading the inspected window does not cause the inspector window to reload. The inspector window can be opened and closed multiple times during the lifetime of a single inspected window, open or closed when the inspected page loads, or open or closed when the inspected page unloads. Furthermore, an inspected page does not necessarily only contain one instance of a promise library, and an inspector window does not necessarily need to limit itself to watching a particular page.

It is not merely conceivable that a promise inspector could watch promises from multiple windows, processes, or instances of various promise libraries, but Causeway is explicitly designed to show the causal relationships between promises in programs distributed over a network.

This implies some basic requirements for a promise inspector protocol for Chrome Extensions.

1. A promise library must be able to communicate with any number of inspectors simultaneously.
2. A promise inspector must be able to communicate with any number of promise libraries, across any number of windows, across any number of processes simultaneously. But for now, just one that will come and go at random, and may or may not be already there when the inspector window loads.
3. Individual promises must have universally unique identifiers. This might imply a crytorandom identifier, or just a sequential number attached to a cryptorandom promise library instance identifier.
4. A promise library must retain a snapshot of all currently pending and rejected but unhandled promises and be able to transmit this snapshot and all subsequent changes each time it connects or reconnects to a promise inspector window.
5. A promise inspector must be able to display stack traces corresponding to where a promise was constructed, their state (pending, fulfilled, rejected, thenable, remote), and their resolution if they are initially pending. The inspector is at liberty to use this information for a variety of visualizations.

Also, the message transport mechanism between an inspected window and an inspector window, with the agents coming and going, is roughly as reliable as any shared network. It might become necessary to construct something like TCP/IP over message ports so multiple objects within multiple contexts can reliably communicate over bidirectional streams.

Do you envision promise libs as having to change to create specific hooks for the inspector? or do you envision a "standard way" that promise libs would have to operate internally so an inspector can automatically understand/track it?

@getify I do imagine promise libraries will have to add code to track pending promises and unhandled rejections, regardless of whether an inspector has been attached, broadcast a message on the window message port when they are ready, listen for a promise inspector’s ping on the window message port, initiate communication with a snapshot of the current situation and then broadcast events on a message port as promises are created and resolved to other promises.

I'm definitely interested in this. Debugging promises is a huge deal for me and anything that improves the ability to debug promises is a killer feature imo.

Long stack traces in Q (and Bluebird) have been very helpful and how libraries can figure out when handlers are attached is very nice.

Just two things I'd like to note:

• I've personally never had to attach handlers at a point that caused the logging error handler to erronously point out a possibly unhandled rejection. Generally the use case of adding handlers to a promise at a much later point is not that common in my code. From talking to other people this has been their experience too. I'd love to hear about such a case.
• Implementing this from userland is a lot harder than implementing this from the browser vendor perspective.
• It would be really cool if this worked in Node too.

I definitely agree that this protocol is a good idea and letting libraries hook on it is very interesting. It's also worth mentioning at the inspector level that it needs to deal assimilation, multiple libraries (And multiple copies of different or the same version of libraries) and so on.

"Individual promises must have universally unique identifiers. This might imply a crytorandom identifier, or just a sequential number attached to a cryptorandom promise library instance identifier."

Why not rely on object identity and use a WeakMap from the promise object to a sequence number held by the inspector? Then the inspector can map from that to whatever other information it needs, without the sequence number needing to be cryptorandom and without making unnecessary assumptions about the promise libraries (other than that they don't reuse promise objects). Note that this doesn't prevent maintaining information about promises that have already been GCd; they will disappear from the WeakMap but their sequence number will still be usable.

From my experience with Deferreds in Python, the most important things to know about a dropped deferred are the stack traces of where it was created, and where it was fired (i.e. fulfilled or rejected). Twisted unfortunately doesn't keep that information by default which is a huge pain.

@daira The promise library can certainly retain a WeakMap with a sequence number. The promise inspector does not exist in the same process. The library would have to broadcast its unique identifier with the initial snapshot, and use sequence numbers to identify promise instances, as those messages would all be marshaled.

I also have some experience with Twisted. Q does not retain old promises either, at the moment, but it would be straightforward. Q already has some code for unhandled rejection tracking.