Skip to content

Instantly share code, notes, and snippets.

@chrisdickinson chrisdickinson/ Secret

Created Nov 10, 2017
What would you like to do?

esm, hurk

walking through the codebase

  • list indention means "I stepped into the code here"
  • TKTKTK means "to come", borrowed from isaac, I have no idea what it really means
  • I've broken these down into sections, the toplevel list items are "entry points" into a file
    • from there, list items in order mean "this executed in this order"
  • another note for background: anytime you see something like "FrobnicatorWrap", that is a strong indication that there's a binding from JS to C++ backing the object. Word to the wise!

Entry Point - entry point

  • config_experimental_modules
  • config_userland_loader - InitConfig - configuration
  • config_experimental_modules -> JS-visible experimentalModules
  • config_userland_loader -> JS-visible userLoader
  • following experimentalModules:
    • lib/internal/bootstrap_node.js L103: only used in bootstrap to issue a warning about how truly experimental it is
    • lib/module.js
      • L451, Module._load -- if we don't have an ESMLoader yet, create one
      • Loader comes from internal/loader/Loader.js (oh lord capcase)
      • We use loader to load the userland loader hooks
      • then we continue to load using getURLFromFilePath
        • this uses the WHATWG URL to create a file:// url
      • we call Loader#import
      • ∅ _load exits


exports a Loader class, which inherits from null (a very bradley thing to do)

Loader entry points:

  • constructor: covered in the type def
  • hook({resolve, dynamicInstantiate}): sets a custom resolver and dynamicInstantiate
  • import(specifier, parentURL = this.base): this seems load-bearing, if you'll excuse the pun!
    • await this.getModuleJob(specifier, parentURL) returns a ModuleJob
      • calls this.resolve (usually ModuleRequest.resolve) L101
      • checks to see if we've got an outstanding ModuleJob in our moduleMap
        • if so, return it
        • getModuleJob exits (great job all)
      • resolve returns a url and a format
      • if the format is mundane (i.e., "not dynamic") pick the loader instance off of ModuleRequest.loaders
      • otherwise create a dynamic loader instance
      • pass the loader instance to ModuleJob
      • END result, returns a ModuleJob
    • gets a module from
    • returns module.namespace()

module request

entry points:

  • static loaders: loaders are provided for:
    • esm -- uses new ModuleWrap (see module wrap)
    • cjs -- uses createDynamicModule (from moduleWrap) (see dynamic loads)
    • builtin -- ditto
    • addon -- ditto
    • json -- surprisingly, ditto
  • static resolve: a Resolver
    • checks for builtins,
    • searches for the file,
    • checks the extension if it exists,
      • throws an error if it doesn't

module job

Entry points

  • constructor
    • constructing a ModuleJob immediately attempts to link all dependend modules (via ModuleWrap#link (see link in module wrap))
  • run
    • calls this.instantiate()
      • visits all modules in dependency graph, adding them to a set
      • once all modules have been visited, calls ModuleWrap#instantiate (see instantiate in module wrap)
        • This instantiates all modules in the graph!
      • then we do some bookkeeping by setting all of our dependency's instantiated props to a resolved promise
    • calls evaluate in module wrap

module wrap

The C++ binding to V8.

  • new ModuleWrap(source, url) - compile a module
  • ModuleWrap#link(resolver : Function)
    • Iterate over all direct dependencies of the current module
    • for each dependency, call anon async fn on lib/internal/loader/ModuleJob.js#L33
      • this recursively creates ModuleJob objects, linking them
      • internally we populate a map of strings to promises for modules (resolve_cache_)
  • ModuleWrap#instantiate()
    • calls V8's InstantiateModule with ResolveCallback
      • resolve callback is called by v8, and it expects a module to be returned
      • this is how we look up modules!
  • ModuleWrap#evaluate()
    • This actually executes the module!
  • ModuleWrap#namespace()
    • This returns the exported namespace of the module.

dynamic loads

Anything that's not ESM is dynamic. There are specializations for cjs, builtin, json, and addon modules (ref static loaders in module request). These specializations use dynamic modules!

See createDynamicModule in lib/internal/loader/ModuleWrap.js.

  • We perform a magic trick.
  • First we create a module that exports an executor variable
  • The module's completion value (that is, the last value evaluted) is a function returning an object with a setter for the executor variable, and a reflect object with getters and setters for individual exported names.
  • We immediately instantiate this facade module.
  • Then we evaluate it, returning the function exposing the executor and exports setters.
  • THEN we create another module that imports "executor" and other exports from ""
    • it also exports those variables
    • and if "executor" is a function, we call it
      • calling executor will mutate those variables so we end up getting the right values
  • We link this fronting module with a resolver that always returns our facade.
  • We instantiate it and send it out into the world.


class Loader

Optionally takes base.

  • moduleMap : ModuleMap: a Map of strings to ModuleJobs
  • base : String: defaults to the file:// url for the CWD
  • resolver : Resolver: a Resolver returning a promise for a url & format
    • if userLoader is not set, this will be ModuleRequest.resolve
  • dynamicInstantiate : Function: for "dynamic" loads

class ModuleJob

Takes a loader, url, and moduleProvider (a ModuleProvider).

  • loader : Loader: The loader that created the job via getModuleJob
  • error : null:
  • hadError : Boolean:
  • modulePromise : Promise<{module : ModuleWrap, reflect : null | ???}>:
  • module : undefined | ModuleWrap: Populated with a ModuleWrap once linked.
  • reflect : undefined | ????: Populated once linked. Unclear what reflect is for.
  • linked : Promise<Array<ModuleJob>>: a promise for the graph of modules depended upon by the current module
  • instantiated : undefined | Promise<>:

fn ModuleProvider

ModuleProvider ::= (
  url : String
) -> Promise<{
  module : ModuleWrap,
  reflect : null | ???

fn Resolver

Resolver ::= (
  specifier : String,
  parentURL : String,
  defaultResolve : Resolver = ModuleRequest.resolve
) -> Promise<{
  url: String,
  format: ValidFormt

summarize (tl;dr)

  • one major cutting point in Module._load
  • provide support for .coffee et al by adding a custom loader
    • no support for doing this at runtime, which seems a little prescriptive
  • dynamic imports support cjs, json, addons, etc
  • in ESM as implemented, require is already undefined
    • we can, in fact, reserve nodejs as a special module name
    • we should be able to "import {require} from nodejs" using the facade pattern

This comment has been minimized.

Copy link

bmeck commented Jan 13, 2018

no support for doing this at runtime, which seems a little prescriptive

This is by design due to limitations around conflicts of mutating the loader system at runtime. It also mimics problems with replacing service workers etc. in the browser.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.