The menu is omakase

omakase.md

Rails doctrine: The menu is omakase (translation: I'll leave it up to you)

Preface

I'm going to try to make the case that instead of focusing on Module requirements, we should focus on Loader requirements and let a thousand flowers bloom. Part of making my case, I will introduce a number of new use cases that may have here-to-fore seemed out of reach.

Parallel Evolution

Once upon a time, when Rails was young, XML ruled the web, at least for API interfaces. Unsurprisingly, XML support was baked into early versions of Rails.

Times change, and now JSON is in vogue.

In a parallel universe, Node was created. The Web Platform (including JavaScript) at the time didn't have modules, buffers, promises, streams, or even a URL definition. By necessity, Node created each of these.

Rails eventually jettisoned XML support. Oh my god, I can hear you thinking, that will break users. Except it didn't. The code was extracted into the activemodel-serializers-xml gem. Developers that wanted to continue to access this function simply had to install and include the new gem.

Node now has two definitions of URLs, one (non-standard) definition of streams, two definitions of the file system interface (one with, and one without promises), and is headed to having two module definitions. And now it looks like we will be heading towards having both process.nextTick() and queueMicrotask.

That's suboptimal, and that's putting it midly.

Brittleness

The current state of Node is that very little can be changed as any change that might break somebody undoubtedly will break a lot of people. This is particularly true for the module subsystem. Consider the following return statement. It is not legal JavaScript, but it clearly works.

This makes implementing the modules features list somewhat impossible. It is impossible to be 100% backwards compatible with CommonJS and 100% compatible with browsers. At least not simultaneously.

So, let's make that a non-goal.

Chroot jails

If you contemplate what import or require do, they do for JavaScript what the file system does for POSIX applications. Many years ago, people realized that one could modify the environment in such a way that subprocesses could operate from a different root directory.

And thus, chroot was born. This begat containers and docker and kubernetes and the cloud.

Imagine an architectural boundary (say, perhaps Workers), across which different code could use a different loader. Different loaders could expose different globals, different modules, and even different semantics entirely for import statements.

This could make it easier to mock and polyfill - without changing code.

And if we create the ability for modules to hide and even proxy other modules, we can enable the creation of secure sandboxes - all without affecting the performce of the underlying modules for people that don't need this level of isolation.

If we would ever want to collapse once again down to a single implementation of fs (the Promisfied version, naturally), and still allow people to opt-in either temporarily or permanently to the classic fs implementation, then the Node core would need to be use a different loader than code in user space.

Issues

"Out of scope" and "technically challenging" would be valid criticisms of this proposal. To which I observe, while definitely hard, implementing this proposal is easier than the impossible task of merging of incompatible module semantics with the added constraint of not breaking anyone.

Imagine a loader that is bug-for-bug compatible with CommonJS (possibly with babelify thrown in). And another loader that is meticulously spec compliant. And one or more that quixotically attempt to find the sweet spot between the two.

Those aren't the problem that this proposal intends to solve. Instead it proposes to focus on the APIs needed to enable the creation of these loaders. And the user interface by which one specifies the initial loader - I'm of the opinion that command line arguments alone aren't going to cut it.

Upsides

So far, I've focused on issues and complexities. That's not the right place to start. The right place to start is with the type of future we want to enable. Specifically:

• A future where there is only one core definition of URL. One core definition of fs.
  • A corollary to this is a future where we can reasonably discuss retiring APIs that no longer are relevant or have been effectively obsoleted by another API.
• A future where node and the web platform continue to evolve separately but together, as opposed to continuing to diverge over time.
• A future where developers can opt-in early to new node core features without upgrading node itself, or opt to continue to use specific features after an upgrade, either temporarily or permanently.

Pre-requisites

• An obvious pre-requisite for this is a pluggable loader.
• Not as obvious, but I strongly believe that this will require the ability to define a persistent configuration, i.e., more than command line options and NODE_OPTIONS provides. At the same time, I would suggest that we co-opt and take ownership of NODE_ENV, enabling separate configurations by environment. Note that configurations in Rails are emitted by generators, and consist of running code. Normally nobody needs to touch these files except for two cases:
  • They have a specific problem that need to address. The generated code has comments that help people find the configuration option and make the correct choices.
  • When upgrading to a new release of Rails.
• Part of this persistent configuration will be the ability to do the equivalent of the current -r or --require for modules that only consist of side effects (doing things such as extending existing classes or modifying globals). This will make it easier for applications to do such things as opting-in to queueMicrotask() with Node version 8, and opting-in to continue to have access to process.nextTick() with Node version 14.