Dynamic module loading done right

module-loading.md

Dynamic module loading done right

Follow-up to Top-level await is a footgun – maybe read that first

Here are some things I believe to be true:

1. Static module syntax is beneficial in lots of ways – code is easier to write (you get better linting etc) and easier to optimise (tree-shaking and other things that are only really possible with static syntax), and most importantly, faster to load (it's trivial for a module loader to load multiple dependencies concurrently when they're declared with a static syntax – not so with imperative statements like require(...) or await import(...)).
2. App startup time is perhaps when performance is most critical. (You already know this, I don't need to cite the studies.)
3. If you're in favour of constructs that jeopardise app startup time, you are anti-user. Top-level await is such a construct.

But the primary motivation for top-level await – loading modules dynamically depending on the environment – is an important and legitimate one. How do we support it with static syntax?

With config. Here's some pseudo-code:

<script>
  System.paths = {
    d3: 'https://unpkg.com/d3@5',
    utils: isModernBrowser() ? '/js/utils-modern.js' : '/js/utils-legacy.js'
  };
  
  System.import('/app.js');
</script>

In this example, we have all the power we need to load modules conditionally, without sacrificing the benefits of static syntax.

The difference of course is that some fourth-order transitive dependency can't conditionally load different versions of itself unless you've configured that in your paths config as well. But that's a good thing! The last thing you want to worry about when developing an app is that some utility from the dark corners of npm that you didn't even know about has the power to bork up your otherwise carefully-engineered app loading experience.

A more advanced variant of this approach would involve a function hook that resolves a module ID programmatically.

You can still load modules dynamically with this approach

When the user clicks on the 'about' page, you can still load it asynchronously:

async function loadAboutPage () {
  const view = await import('/js/views/about.js');
  view.render( document.querySelector( 'main' );
}

If you're into PRPL then of course you can also prefetch that module when you get a spare moment. The point is, this is purely about not introducing ways for modules to sludge up the initial load. Top-level await doesn't give us any new expressive power, it just introduces slightly nicer syntax but with a real cost.

Between JavaScript modules and HTTP2, we have the potential to make it really easy to write fast-loading apps. Top-level await could undermine that. Let's not do it.

God, thanks! I've been wondering about how to do dynamic loading and this fits my use-cases just fine.

I wonder if it matters tho. It seems modules simply are not anywhere near ready. And not clear the System object will ever exist, with config. Certainly the initial module implementation with be <script type = module ...>. God knows what node plans, they certainly have argued for quite a while (.jsm??).

@backspaces yep, this is all pseudo-code – am just trying to outline a general approach that addresses the reason people wanted top-level await in the first place

the original loader spec had the ability to do custom loading functions and was flexable enough to let me write a loader that worked for about 99% of commonjs modules, while I haven't kept up to date with the current spec it seems to have a resolve hook still which would allow subbing in your own module dynamically, meaning this could probably be done with the current loader api

My 2cent here are easy explained. We Simply need to be aware that await blocks in its context and this is not evil and will not be any problem in real world for example if you dynamic import() a module that uses topLevelAwait it will still don't block rendering. as you pointed out its only for conditional exports. thats why it is top level await and there are tons of other ways to block execution already if you import a bad coded lib there is nothing you can do about it.

Conclusion

• top level await is nice just let it be
• blocking without the need to do so is bad don't do it. Remember any Sync Action is blocking. In General not only await.
• if top level await is evil we should talk about sync file access like needed for http servers when they use none hard coded cert files :)