Skip to content

Instantly share code, notes, and snippets.

@littledan
Last active June 29, 2022 01:07
Show Gist options
  • Star 3 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save littledan/f7c1d1abf0e51ad4b526a8eadb2da43b to your computer and use it in GitHub Desktop.
Save littledan/f7c1d1abf0e51ad4b526a8eadb2da43b to your computer and use it in GitHub Desktop.
Anonymous inline modules

Note: this document is subsumed by the module blocks proposal

Anonymous inline modules

Anonymous inline modules are syntax for the contents of a module, which can then be imported.

let inlineModule = module {
  export let y = 1;
};
let moduleExports = await import(inlineModule);
assert(moduleExports.y === 1);

assert(await import(inlineModule) === moduleExports);  // cached in the module map

Importing an anonymous inline module needs to be async, as anonymous inline modules may import other modules, which are fetched from the network.

let inlineModule = module {
  export * from "https://foo.com/script.mjs";
};

Anonymous inline modules are only imported through dynamic import(), and not through import statements, as there is no way to address them as a specifier string.

Syntax details

PrimaryExpression :  InlineModuleExpression

InlineModuleExpression : module [no LineTerminator here] { Module }

As module is not a keyword in JavaScript, no newline is permitted between module and {. Probably this will be an easy bug to catch in practice, as accessing the variable module will usually be a ReferenceError.

Realm interaction

As anonymous inline modules behave like module specifiers, they are independent of the Realm where they exist, and they cannot close over any lexically scoped variable outside of the module--they just close over the Realm in which they're imported.

For example, in conjunction with the Realms proposal, anonymous inline modules could permit syntactically local code to be executed in the context of the module:

let module = module {
  export o = Object;
};

let m = await import(module);
assert(m.o === Object);

let r1 = new Realm();
let m1 = await r1.import(module);
assert(m1.o === r1.o);
assert(m1.o !== Object);

assert(m.o !== m1.o);

Use with workers

It should be possible to run a module Worker with anonymous inline modules, and to postMessage an inline module to a worker:

let workerCode = module {
  onmessage = function({data}) {
    let mod = await import(data);
    postMessage(mod.fn());
  }
};

let worker = new Worker(workerCode, {type: "module"});
worker.onmessage = ({data}) => alert(data);
worker.postMessage(module { export function fn() { return "hello!" } });

Maybe it would be possible to store an inline module in IndexedDB as well, but this is more debatable, as persistent code could be a security risk.

Integration with CSP

Content Security Policy (CSP) has two knobs which are relevant to anonymous inline modules

  • Turning off eval, which also turns off other APIs which parse JavaScript. eval is disabled by default.
  • Restricting the set of URLs allowed for sources, which also disables importing data URLs. By default, the set is unlimited.

Modules already allow the no-eval condition to be met: As modules are retrieved with fetch, they are not considered from eval, whether through new Worker() or Realm.prototype.import. Anonymous inline modules follow this: as they are parsed in syntax with the surrounding JavaScript code, they cannot be a vector for injection attacks, and they are not blocked by this condition.

The source list restriction is then applied to modules. The semantics of anonymous inline modules are basically equivalent to data: URLs, with the distinction that they would always be considered in the sources list (since it's part of a resource that was already loaded as script).

Optimization potential

The hope would be that anonymous inline modules are just as optimizable as normal modules that are imported multiple times. For example, one hope would be that, in some engines, bytecode for an inline module only needs to be generated once, even as it's imported multiple times in different Realms. However, type feedback and JIT-optimized code should probably be maintained separately for each Realm where the inline module is imported, or one module's use would pollute another.

Support in tools

Anonymous inline modules could be transpiled to either data URLs, or to a module in a separate file. Either transformation preserves semantics.

Named modules and bundling

This proposal only allows anonymous module definitions. We could permit a form like module x { } which would define a local variable (much like class declarations), but this proposal omits it to avoid the risk that it be misinterpreted as defining a specifier that can be imported as a string form.

Anonymous inline modules proposal has nothing to do with bundling; it's really just about running modules in Realms or Workers. To bundle multiple modules together into one file, you'd want some way to give specifiers the inline modules, such that they can be imported by other modules. On the other hand, specifiers are not needed for the Realm and Worker use cases. This inline modules proposal does not provide modules with specifiers; a complementary "named inline modules" proposal could do so. Note that there are significant privacy issues to solve with bundling to permit ad blockers; see concerns from Brave.

@littledan
Copy link
Author

This discussion about file: is completely off topic from the topic of this gist. The gist does not fix file:. There are legitimate security issues around file:.

@guest271314
Copy link

Asked about file protocol for testing purposes. https: is far more vulnerable re "security issues" than file: protocol.

There are legitimate security issues around file:.

What evidence do you have to substantiate that claim? w3c/webappsec-secure-contexts#66

@guest271314
Copy link

@littledan A case where this will be useful is initiating an AudioWorklet instance without having to make a network request WebAudio/web-audio-api-v2#109.

@guest271314
Copy link

@littledan @ljharb It should be possible to load an Ecmascript module without making a network request. For example, when trying to construct an AudioWorklet at console at github.com CSP errors will be thrown. In this case we already have the code. The only non-essential restriction is AudioContext.audioWorklet.addModule() which makes a fetch request where we do not need to make a fetch request. I worked around this on Chromium using DevTools Local Overrides. However, it should be possible to run AudioWorklet at console on GitHub without making a network request where we already have the code inline.

I am banned from WHATWG/HTML and evidenbtly TC39 as well WebAudio/web-audio-api-v2#109 (comment)

This should be talked about at the EcmaScript and Worklet level before considering using it in Web Audio, this is not really a Web Audio API issue.

so I have no way other than this gist to communicate this matter to you folks.

@guest271314
Copy link

I sent this proposal to es-discuss

Inline module without network request

Web Audio API uses Ecmascript / WHATWG/HTML modules for AudioContext.audioWorklet.addModule(). That means that a network fetch request must be made to construct an AudioWorklet node.

When we already have the code we do not need to make a network fetch request however the design of addModule() demands that a request be made to construct the AudioWorklet node.

If we are a console on GitHub or any other site that has CSP restrictions we cannot construct an AudioWorklet for the purposes of processing and outputting audio even though we can make a quic-transport protocol request.

This proposal is for a means to load a Ecmascript / WHATWG/HTML module without making a network request where we already have the code inline.

In practice the code can be as simple as

AudioContext.audioWorklet.addModule(code, {inline: true})

where {inline: true} configures the module loader to interpret code as raw JavaScript code instead of a URL, negating the need to make a network request.

References:

Alternatives for module loading of AudioWorklet #109 WebAudio/web-audio-api-v2#109

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment