Use of IDs instead of URLs for HTML resources

gistfile1.md

Design forces:

• ECMAScript (ES) Module Loader API is coming. The ES module ID syntax are strings that are just treated as string IDs and not paths.
• Existing JS module experience in CommonJS/Node/AMD worlds of using module IDs that are resolved to a URL/path for fetching.
• Better package managers are coming for front end web apps. These package managers install assets by IDs not by paths.
• For front end code, baseURL + moduleID + '.js' is likely the default ID-to-URL resolution, but other declarative config could be possible for browser-based ES Module Loaders. Examples of useful declarative config in this area are the problems solved via common AMD loader config

So it will be common in JS code to use string IDs, and not URLs to refer to dependency resources.

With the coming of web components and custom elements, it will be possible for a custom element to depend on other custom elements.

For web components HTML Imports are used to try to express the dependencies. However, it suffers from a few deficiencies when coming from the world of a JS module dependency mindset:

• HTML Imports use URLs to reference the import
• By forcing the HTML Import for dependencies to be in the custom element's definition, it makes it harder to share the same custom element dependency across custom elements: they would all have to guess the right path. This is harder to do when a package manager installs dependencies.
• custom elements name themselves. In JS module space, this is an anti-pattern: modules do not name themselves, but derive their name from how other modules refer to them, or how they are installed by the package manager. This is a very useful feature, allows swapping in implementations, even different implementations depending on what ID wanted it.

What I would like to see are two things:

1. Custom element resolution to pass through either the current global ES Module Loader instance, or for the DOM to have an equivalent Loader object that allows for at least the normalize and resolve steps that ES Module Loaders have.
2. In general, allow HTML dependencies to use module-like IDs instead of URLs. These IDs would be given to a Loader for normalize and resolve steps to produce a URL that the browser could then use to find the resource.

Custom element resolution

When the browser parses a DOM fragment and finds a custom element it does not know about, it asks a Loader object to resolve that element ID to a URL. Example:

When the browser sees 'some-thing', ask a loader to resolve that to a URL, and do the HTML Import-style of work without the HTML code above having to have an explicit HTML Import tag in the source.

//JS psuedocode for what browser does:
var url = System.resolve('some-thing');
// create an HTML Import using that URL, load it,
// then create some-thing custom element.

Furthermore, I would want the ability for the URL that was created to be a JS module, instead of an HTML fragment. This would allow JS dependencies for the imported custom element.

If a JS module, then the module export is the object used for the document.register() call. This would avoid the module needing to name what tag it is for. Recall that modules (and custom tags) should not name themselves.

There is a bit of grey area on how to bind a template element for that module export that is used for the document.register call, but my first impulse is to use an AMD loader plugin-style dependency like 'template!some-thing.html'.

It may be enough though, from the HTML spec side, to just say that "the object passed to document.register has a template node attached to it".

ID resources instead of URLs

So, anywhere a URL is used in an HTML attribute, an "id" form could be used instead. Those IDs would pass through to the Loader.resolve() to get a URL that can be used by the browser.

Some existing URL-based attributes, and their ID-based equivalents:

• href -> hrefid
• src -> srcid

IDs for these would be of the form ID + '.' + extension. So, to reference a "boostrap" CSS file for a dialog:

<link rel="stylesheet" hrefid="boostrap/dialog.css" type="text/css">

The bootstrap/dialog would be passed to Loader.resolve() to get an URL path, and the extension would be added to the end of that path to get the final URL.

There is a restriction that the last dot is treated as a file extension, but I think that is a fine restriction. If you want to be more formal, feel free to specify a specific separator to indicate any file extension. Maybe just .., so bootstrap/dialog..css. For AMD require.toUrl() though we just live with the restriction of the last dot indicating file extension and strip that off to get the ID.

Main points

The above are just sketches. The main points are that there should be non-URL pathways to specify resources, and some ID-to-URL resolution. This is particularly important once dependencies are in play.

The main analogy is how ES modules have moved away from direct URL references and use IDs instead, and also, moved away from correctly ordered <script src=""> tags for dependency resolution. The same should be possible for non-JS dependencies.

If I had written this myself (and I soon would have), it would not have differed substantially.

@bkardell mentioned on twitter: 1) HTML imports are designed to be useful outside custom elements. 2) would there be a tag to configure hrefid mappings or just imperative in a script:

1. That is fine, but using them for scripts in particular make me cringe, if there is an ES module loader already available. The changes above though would still allow HTML Imports as a concept to exist, maybe used under the covers by the browser after resolving the ID in the custom element resolution case.

2. I would say no, at least not initially, maybe never. It should be done in script. I do not mind seeing a declarative object syntax to configure the loader, but imperative hooks should be at the very bottom of the API.

re #2, "imperative at the bottom"... indeed!! Explain the magic, fundamental primitives, etc... That wasn't really my question though... it seems that this config, like a lot of things that you'd want to share that across your site, so then what? Move it into a script, maybe flag it with preloader hints and have it execute in the global space? I mean, you could do that (if so I think you should include in examples above), but it feels like you want a sugared pattern to load this config, kinda like a manifest - that is why I suggested maybe <link type for loading and configuring with that information.

@bkardell: right I think there is tension between use cases: For me, I usually want JS to be the first entry point into the page construction, so the JS file containing this config would be loaded before the choice of UI/HTML markup to show would be shown. So I would load the config async in JS before deciding what HTML tags to inject into the page.

However, I can see where a more static page that always had a known UI start point would benefit from a load of the config only before HTML parsing started.

In that case though, I can see just using a plain, <script src=" tag to load the config, and done as a script tag that blocks rendering, no async/defer on it.

I completely agree with the core arguments, and like the ideas. I must say I am not too clued up on the intricacies of custom element registration, so can't comment too heavily on this.

With the hrefid, it might also be necessary to specify the name of the loader to use. The loader system allows custom loaders stored as globals. Perhaps a way to indicate the global loader name alongside the id would be necessary here.

I do think though that while it is not ideal, it would be possible to use HTML imports properly with external dependencies, alongside modules, in its current form.

I think the important problem is understanding how HTML imports and ES6 modules work together. It would be incredibly naive to imagine that HTML imports solve all dependency problems in the browser. At some point it will need to be used alongside ES6 modules, and working some of those parts out now is important.

In this scenario my worry is how the loading cycle is determined.

Allowing HTML imports to work with external dependencies

It seems that the fundamental issue is that HTML imports struggle with the concept of "external dependencies". If two HTML imports both need jQuery through an HTML import, who decides what the absolute URL is to jQuery?

If I want to install a component just by dropping in its HTML import, I have no control over these absolute dependencies.

Then we have versioning problems such as whether the version gets used in the URL? And if not, what happens if libraries need different versions of an external dependency?

The best way to manage this would be a convention like the following:

1. Imports always backtrack down to below their root folder to look for a dependency.
2. Imports add a version suffix to the dependency folder they are looking for.

So an import located at "packages/bootstrap@3.0/bootstrap.html" would contain an import to jQuery looking like:

<link rel="import" href="../jquery@2.0/jquery.html" />

Note that minor version only must be used here, because it would be a lot to expect all dependencies to match an exact revision. At the global level, the latest revision should be used.

An ecosystem of HTML imports could work like this, solving the external dependency problem. But it certainly doesn't have the elegance or simplicity of the module resolution system.

Getting HTML imports to play well with ES6 Modules

Do we really expect an HTML file of the form:

jquery.html:

<script src="jquery.js"></script>

To be the long-term solution to dependency management in the browser?

At some point users will just want to write:

bootstrap.js:

  import $ from 'jquery@2.0';

At the surface they are largely orthogonal. While styles in HTML imports block rendering, scripts don't, so having an import (which is by its nature asynchronous) wouldn't alter the functional behaviour of the loading cycle at all.

But the tricky part is working out when the page has loaded.

Consider:

bootstrap.html:

<link rel="stylesheet" href="bootstrap.css" />
<script>
  System.import('jquery', function($) {
    // do stuff
  });
</script>

How do we now know when bootstrap.html has imported jQuery and finished setting itself up?

In this case, I think the most imperative feature to allow HTML imports to work together would be a system for loading modules in a script, and allowing that to affect the readiness of the page:

bootstrap.html:

<link ref="stylesheet" href="boostrap.css" />
<script loader=System>
  import $ from 'jquery@2.0';
  // do stuff
</script>

We then would only trigger the "loaded" event for boostrap.html once the script has finished its thing.

Even more ideally, we could have two loading events: one for the script and one for the style.

Summary

1. hrefid and srcid would allow the module system to work with HTML Imports really nicely, although the loader name may also be necessary here.
2. Without this, deep backtracking would still allow modular HTML Imports, but it would need to be imposed as a convention and globally used by everyone.
3. I tend to think the most important feature in having HTML Imports and ES6 Modules working together would be module loading in <script> tags, and support for the load events of these in the browser. Hopefully we can learn the lesson from domready and not end up with a complex loading detection situation.