README.md

Toward pure dynamic module loading in XS

require() is synchronous. bundleScript includes all of rollup and exposes the text form of the module at runtime, which seems like an awkard API.

I'm inclined to focus on import(): it's asynchronous and returns just the module exports.

Eventual send tildot: TODO

I haven't worked out the details of tildot expansion on xs, but I do know where the xs C API hooks are for getting the module source with fopen() and such... It seems simplest to require that dynamic modules have tildot pre-expanded somehow.

Limited access to import()

import() inside a Compartment is limited to the modules in Compartment.map. We can pass a s => import(s) function from the primordial realm into a compartment, but that could allow the compartment to load powerful modules.

XS provides an fxFindModule hook. After primordial realm initialization, we can set a flag so that it doesn't find powerful modules. It seems safest to have a clear naming convention to distinguish powerful from pure modules; for example, only module locations starting with /vat/ are allowed as pure.

Example of how not to do it

This example shows vat code getting access to the filesystem. But it also shows where we could set a flag to prevent this:

connolly@jambox:~/projects/moddable/examples/modfun$ mcconfig -o ./build -m -d -p x-cli-lin && ./build/bin/lin/debug/modfun 
# xsc main.xsb
# xsl modules
# xsl modules
# cc mc.xs.c
# cc modfun
lin_xs_cli: loading top-level main.js
@@fxFindModule: main
@@fxFindModule: mc/config
@@fxFindModule: file
 lin_xs_cli: loaded
lin_xs_cli: invoking main(argv)
in primordial main(). config.pwd: /home/connolly/projects/moddable/examples/modfun Compartment.map keys: Resource,instrumentation,timer,main,capmain,file,mc/config
Primordial realm is done loading modules. We should set a flag so fxFindModule no longer find powerful modules.
pureModules: Resource,main,capmain
@@fxFindModule: capmain
in capmain.main(/home/connolly/projects/moddable/examples/modfun). Compartment.map keys: Resource,main,capmain
@@fxFindModule: /home/connolly/projects/moddable/examples/modfun/gameVat.js
@@fxFindScript: /home/connolly/projects/moddable/examples/modfun/gameVat.js
@@fxLoadScript: /home/connolly/projects/moddable/examples/modfun/gameVat.js
@@fxFindModule: file
@@fxFindScript: /home/connolly/projects/moddable/examples/modfun/file.js
 lin_xs_cli: main() returned a promise; entering event loop
imported dynamic vat.
eek! vat got secrets! root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin
bin:x:2:2:bin:/bin:/usr/sbin/nologin
sys
dynamic vat setup: game setup stuff
capMain done.
primordial main done.

capmain.js

/* global trace, Compartment */

const { keys } = Object;

export async function capMain({ loader, pwd }) {
  trace(`in capmain.main(${pwd}). Compartment.map keys: ${keys(Compartment.map)}\n`);
  try {
    const { setup } = await loader(`${pwd}/gameVat.js`);
    trace('imported dynamic vat.\n');
    const stuff = setup();
    trace(`dynamic vat setup: ${stuff}\n`);
  } catch (oops) {
    trace(`oops! ${oops.message}\n`);
  }
  trace('capMain done.\n');
  return 0;
}

gameVat.js

import { File } from 'file';

export function setup() {
  const secrets = new File('/etc/passwd').read(String).slice(0, 120);
  trace(`eek! vat got secrets! ${secrets}\n`);
  return 'game setup stuff';
}

main.js

/* global trace, Compartment */
import config from 'mc/config';
import { File } from 'file';

const { keys, freeze } = Object;

const powerful = freeze(['file', 'instrumentation', 'timer', 'mc/config']);

export default async function main() {
  trace(`in primordial main(). config.pwd: ${config.pwd} Compartment.map keys: ${keys(Compartment.map)}\n`);

  trace(`Primordial realm is done loading modules. We should set a flag so fxFindModule no longer find powerful modules.\n`);
  const pureModules = { ...Compartment.map };
  for (const s of powerful) {
    delete pureModules[s];
  }
  trace(`pureModules: ${keys(pureModules)}\n`);

  const { capMain } = new Compartment('capmain', {}, pureModules).export;
  try {
    await capMain({ loader: s => import(s), pwd: config.pwd });
  } catch (oops) {
    trace(`primoridail.main: oops! ${oops.message}\n`);
  }
  trace('primordial main done.\n');
  return 0;
}

manifest.json

{
  "include": [
    "$(MODDABLE)/examples/manifest_base.json",
  ],
  "modules": {
    "*": [
      "./main",
      "./capmain",
      "$(MODULES)/files/file/*",
      "$(MODULES)/files/file/lin/*"
    ]
  },
  "config": {
    "pwd": "/home/connolly/projects/moddable/examples/modfun"
  }
}