Node.js mocking example

mock-loader.mjs

const mocks = Object.create(null);
global.mock = function (_mocks) {
  Object.assign(mocks, _mocks);
};

export async function resolve (specifier, context, parentResolve) {
  if (mocks[specifier]) {
    return { url: 'mock:' + specifier };
  }
  return parentResolve(specifier, context);
}

export async function getFormat(url, context, parentGetFormat) {
  if (url.startsWith('mock:')) {
    return { format: 'dynamic' };
  }
  return parentGetFormat(url, context, parentGetFormat);
}

export async function dynamicInstantiate (url) {
  const obj = mocks[url.slice(5)];
  const exports = Object.keys(obj);
  return {
    exports,
    execute (ns) {
      for (const key of exports)
        ns[key].set(obj[key]);
    }
  }
}

run.sh

node --loader ./mock-loader.mjs test.mjs

# Alternatively, locate at node_modules/mock-loader/loader.mjs with a package.json "exports": "./loader.mjs" set
# Then use something like:
# NODE_OPTIONS="--loader mock-loader" node test.mjs

test.mjs

mock({
  express: {
    custom: 'express'
  }
});

import('express').then(x => console.log(x));

I don't think this is enough. Mocking should be dynamic, i.e. a developer might like to implement a mock with some assertion for each tests.

As an example, consider https://github.com/mcollina/sonic-boom/blob/a6798ae1b956484c0cbce20df2e249a1d60d97c6/test.js#L488-L501.

@mcollina sure I've updated the example to demonstrate dynamic mocking.

I don't understand how that is dynamic. From my point of view, dynamic would mean that I can override the content of a module from within another, similarly to how proxyquire works.

@mcollina if you're asking for the explicit import() call itself to have its own registry with stubs, such that those stubs can be forgotten and rerun with new stubs, then the best approach to that problem would be to create "sub loaders". Similarly to how SystemJS allows new loader instances to be created, or how the Realms API encapsulates loader creation. Right now in Node.js we don't have the ability to instantiate hooks into a new registry / realm - I think it would be best to follow the Realms API here. The VM module does provide the base functionality though to achieve these goals, but would take a little more userland wiring to put together a full proxyrequire.

If you would be interested in sponsoring a VM-module-based proxyrequire approach my consulting company can certainly quote for a contract on it to NearForm, just let me know if you would like to discuss further.

This stem from a discussion with Myles on having mocking support for esm before marking it stable, as it’s still not on par with cjs. Thanks, I think I have my answer.

Unfortunately I do not have any budget to sponsor this endeavor (yet).

That is a shame, because I would be happy to help if compensated.

@mcollina after seeing your comment about not being so concerned about a full graph solution, I put together a new concept, which may align with what you were after. Let me know your thoughts. There are a lot of approaches here which is why we see this as something for userland to provide - Node.js core only provides the primitives.

I didn't even think it was possible looking at the primitives in Node.js core :D. Thanks, this is what I was looking for. If you don't mind I'll stick all of that into a module sooner rather than later. How should I attribute?

Great to hear, please feel free to use! No attribution is necessary.

Note there are currently plans to separate loaders into their own thread. If that PR lands the technique here would have to adapt to that, effectively with some extra instrumentation on both ends. The same overall API can be supported fine with the new primitives though, and I can be sure to track that remains possible.

The other thing that might affect this in future is any changes to the dynamicInstantiate hook which is also considered unstable currently. Anything that replaces it would still permit the same functionality as well.

Personally I would like us to separate modules stability from loader stability - loaders are a much more advanced feature that do still need a little time to mature (especially without a lot of active development, I have had to draw a line on this myself without funding unfortunately as mentioned). I hope we could consider stable modules without stable loaders though, and that is an important discussion to have.

In terms of supporting API changes, in theory even as loader APIs mature it should still be possible to write loaders that work in older versions of Node.js using some ugly techniques, but there are possibilities there too.

Note there are currently plans to separate loaders into their own thread. If that PR lands the technique here would have to adapt to that, effectively with some extra instrumentation on both ends. The same overall API can be supported fine with the new primitives though, and I can be sure to track that remains possible.

Will the loader and the main process share two different set of globals? How could the same API keep working if so? The important part is the the fact that any closures that are passed through mock() are retained.

Personally I would like us to separate modules stability from loader stability - loaders are a much more advanced feature that do still need a little time to mature (especially without a lot of active development, I have had to draw a line on this myself without funding unfortunately as mentioned). I hope we could consider stable modules without stable loaders though, and that is an important discussion to have.

My biggest concern is that without mocking capabilities, the only viable approach to testing esm is to use heavy transpilation similar to what jest does, which is not really esm anyway. (The distance of what jest executes vs what node executes is significant).

Will the loader and the main process share two different set of globals? How could the same API keep working if so? The important part is the the fact that any closures that are passed through mock() are retained.

They can't! Rather, the main thread mock call would use a unique identifier that can serialize to the loader thread, that then coordinates that the binding run through the mock call is used for the main thread module instance. That is, the binding is still created, setup and shared on the main thread, but the loader on the other thread has to coordinate that through a serialization interface over having access to the binding explicitly.

My biggest concern is that without mocking capabilities, the only viable approach to testing esm is to use heavy transpilation similar to what jest does, which is not really esm anyway. (The distance of what jest executes vs what node executes is significant).

If you consider the mocking example above enough capability, then I can guarantee that the capability will remain possible despite any future API changes. The API just won't necessarily be stable. It should be possible for a userland mocking API to be created that provides a stable interface to users working over all the unstable versions. Happy to discuss collaboration further there too if you like.

The distance of what jest executes vs what node executes is significant

Can you clarify your point here further?

Can you clarify your point here further?

Jest mocks rely on transpiling instead of using any proper real-time esm mechanism. Essentially they are fundamentally incompatibile with Node.js esm implementation - they sidestep it completey as they are fully transpiling everything.