EDIT There is a second version of this proposal down in the comments. https://gist.github.com/445adc206d51a038de4a#gistcomment-359858
I really like Isaac's module proposal. Having recently designed a module system for the luvit project (node ported to lua), I've thought a bit about this.
I think we can set a few minimal rules and reduce a ton of edge cases. One in particular is cyclic dependencies. I don't think they are worth the pain and should simply be disallowed.
To help understand things, we should make a distinction between dependencies and peers. In a large application or package there are several files containing code. Some files depend on things defined in other files before they can start. Simple dependencies that block startup can't be cyclic or you'll have a deadlock at startup. So the other type of dependency is wanting to reference an api or function from another file during runtime.
The basic syntax I propose is like the one Isaac showed (if I understand it correctly). There are two new keywords, import
and export
. The import
keyword must be followed by a string literal that is the identifier for the file you want to import. The export
keyword is like return
and yield
in that it sends a value. It does not affect control-flow, but it does set the current file's export value.
Consider lib/add.js
:
// lib/add.js
function add(a, b) {
return a + b;
}
export add;
And then we have a peer file that depends on this:
// lib/usesAdd.js
var add = import "./add.js";
var result = add(2, 3);
Now suppose we wanted to write a math package that contains add as well as other functions.
// math.js
export {
add: import "./lib/add.js",
multiply: import "./lib/multiply.js",
// ...
};
Here is where this gets tricky. Suppose that we want to implement multiplication using addition and a loop. So our multiplication library will need to depend on add. I can't depend on the main app because that depends on it. So it has to depend on add directly.
// lib/multiply.js
var add = import "./add.js";
function multiply(a, b);
var product = 0;
for (var i = 0; i < b; i++) {
product = add(product, a);
}
return product;
}
But not we learn later on that add needs to access multiply for some reason. This happens all the time in real applications. How would this be required?
I really want dependencies with import and export to be expressible as a DAG (Directed Acyclic Graph). There is a clear order for loading the executing the files. There are two reasons for this. One, it makes a lot of nasty edge cases with current module systems simply go away. And second, JavaScript is single threaded. Even with the exports.foo = bar
syntax in node and CommonJS, the dependencies have to be executed in some serial order. This may surprise people when you're allowed to require some module, but it's methods aren't populated yet because you were executed before it was. This would be a leaky abstraction.
So then how would I solve this dilemma using acyclic dependencies? I would handle it at the application level where I control the logic and know what to expect.
I would rewrite math.js
as follows:
// math.js
var math = {};
export math;
math.add = (import "./lib/add.js")(math);
math.multiply = (import "./lib/multiply.js")(math);
Then in each module I would export a function that accepts the math
namespace.
// lib/add.js
export function (math) {
return function (a, b) {
// ... might use math.multiply
};
};
There are several other techniques that come to mind that I could employ for other use cases. The point is that using the basic primitives that would be provided by the language, I can organize my app however I want. My code would use the same style in the browser or in node. The browser could pull my code on demand as it's needed or a server-side compiler could statically analyze the imports and generate a single js file that the browser then reads. All JS code everywhere would be written using the same simple module syntax, but without imposing huge constraints on the runtime or providing leaky abstractions to applications.
Thanks for all the feedback everyone. Here is my updated proposal with @isaacs's ideas assimilated inline. I like this much better than my original proposal and think that some level of cyclic dependencies can be baked into the module system by using a caching system like Lua does.
Loader
Loader is a global provided by the runtime. It is not fully implemented by the language, but an interface implemented by things like node or browsers. This provides the runtimes a place to hook I/O for their module system and allows script authors access to do advanced things like dynamic requires or module concatenation.
Loader.preload
An object containing compiled and defined module functions.
Loader.loaded
An object containing cached results from
Loader.preload
. Modules are only run the first time they are required. Calls after that pull the cached value from here.Loader.resolve(requestPath, callerPath, callback) -> fullPath
This function is implemented by the runtime to map require paths to fully qualified paths (still opaque identifiers as far as the language is concerned since JS has no concept of I/O).
Loader.fetch(fullPath, callback) -> contents
This functions is implemented by the runtime. It fetches the code contents and returns it in the callback.
Loader.require(fullPath)
Execute a module and return the result (or cached result). It must be already defined and fully loaded or this will throw.
Loader.define(fullPath, contents, callback)
Defined by the language. This will parse and compile the contents. It will recursivly call resolve, fetch, and define for any dependencies mentioned in the contents. The callback will be called when all is done, but the value
Loader.preload[fullPath]
will be set early on so some form of cyclic dependencies can be satisfied.Here is some pseudo-code to show the in-language part of this interface:
The only keyword added will be
require
. This will be used byLoader.define
to scan for dependencies as well as callLoader.require
at runtime with the resolved paths that were found during define.This proposal works well for node style runtimes, it works for on-demand browser runtimes (where each dependency is requested till enough is loaded to start executing) and pre-compilers can simply concatenate several
Loader.define(fullPath, contents)
calls to prime the IO caches and prevent round-trips.