Skip to content

Instantly share code, notes, and snippets.

@cramforce
Last active May 29, 2017 10:08
Show Gist options
  • Star 4 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save cramforce/e1a8cea7af01161fe752e8b398b55bd2 to your computer and use it in GitHub Desktop.
Save cramforce/e1a8cea7af01161fe752e8b398b55bd2 to your computer and use it in GitHub Desktop.
ES6 modules and bundling

Status quo

Current ES6 module usage (and general JS usage) in the browser relies on transpilation (due to non-widepread support in browsers) and bundling (to limit HTTP request count and HTTP waterfalls) for performance (module count easily goes into the 1000s which even with HTTP2 cannot be efficiently loaded in a module-per-request strategy; even when using Push to avoid HTTP waterfalls).

Bundling strategies

The bundlers use 2 large classes of strategies to bring N modules into a single JS file:

  1. (require.js, browserify, webpack): Each module gets put into a function that can be addressed by name. There is typically an export object per module where all exports are properties of that object.
  2. (closure compiler, rollup): Modules get compiled away and exported symbols become essentially global variables (at least within compilation unit) and directly accessed by other modules.

Approach #1 has the benefit of being simple and allowing easy late loading (late loaded JS can access already loaded modules by name). Its primary downside is that it has performance issues due to the frequent hash lookups during inital execution of JS and indirection when accessing functions from different modules.

Approach #2 undos the performance issues, but it has the problem that interop between late or concurrent loaded modules requires somehow marshalling symbols between files. Rollup does not support this at all. Closure Compiler supports it by exposing all exports to the global scope or by exposing them to a single global object (incurring a property lookup (often hash lookup as this object will have 1000s of properties) cost per access).

Interop

These approaches all share a problem that they cannot interoperate with modules from other ecosystems. E.g. one cannot put up a URL like https://cdn.foocdn.com/react.1.3.5.js that can be shared by all React users (and cached by web users) and load that as an ES6 module making it available to the bundled code as that code no longer has a notion of modules at all. Several modes of hybrid systems can, of course, be envisioned, but these would make the bundlers even more complex and they are already perceived as very complex by devs today.

Proposal 1

Giving the reality of bundling, it would be great if modules could still exist as first class entities when grouped into a single file. One way to do it would be through a type of module keyword like

module 'name/like/in/import' {
   module content
}

module 'module2' {
   module content
}

module 'other/module' {
   module content
}

With this different compilation units gets to interact with one another one the ES6 module level without relying on potentially inefficient and hard to optimize runtime patterns.

Proposal 2

Specify an archive format (jar, zip or similar) that allows retrieving N modules (and potentially) other resouces in a single request. The primary benefit here is independence from JS syntax.

Alternatives

  • Use complex loaders that implement something like the proposal above at runtime invoking an eval-like function for each module.
@Munter
Copy link

Munter commented Sep 11, 2016

AFAIK systemjs can do the thing you say is missing, importing a module through a cdn url

@ItamarGronich
Copy link

Why can't module loaders today use some kind of algorithm to determine exactly how many nodules need to be loaded at a time and issue a request to load them all with a single request?
Why do younthink we need a new format with a new browser API using zip/jar file?

@rektide
Copy link

rektide commented Sep 14, 2016

@ItamarGronich: @cramforce mentions HTTP2/Push, which could permit 0 network requests to be issued. The asserted problem is that pushing many independent files still incurs too weighty cost:

This is not because HTTP2 is slow, it is because there is RAM overhead per request (e.g. a script DOM node) and IPC cost to network thread.
https://twitter.com/cramforce/status/774261066180104193

IPC Cost

Even with Push and zero network request cost (via push), the page still has to have some way to load all the modules: there still need to be 1000's of < script> tags on the page, which is a cost. I'm not sure that price would be significantly weightier than the cost of a bunch of module 'mymodule' {}'s throughout (post-compression). So what is the IPC cost? I think it'd be the header block cost. I'm not sure, in practice, how small this can be made.

RAM Cost

I have no data on this and would like to know more. I personally want to believe, whatever these costs are, they're something browsers could optimize and improve on, but I'm biased & want to believe that.


If the goal is absolute minimum cost, I tend to think there is definitely room for more work to be done for optimization. On the other hand, I think a massive number of webdevs have been very hopeful that we could get beyond bundling and wrap up steps, and start having module == file, which wouldn't eliminate build tool usage (minification, tree-shaking, &c).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment