Fastboot build meeting notes

fastboot-build.md

FastBoot Build meeting notes

Date

02/06/17

Attendees

• Tom Dale
• Robert Jackson
• Stefan Penner
• Ryan Cruz
• Krati Ahuja

Overview

This meeting was primarily to discuss, brainstorm and come up with a solution on how to make fastboot builds more performant. Currently FastBoot 1.0 release is blocked on two major items:

1. Serving the FastBoot base page in dev mode: Currently FastBoot brings up its own express server to serve the base page and other assets. This does not play well when build is broken due to transpilation, no livereload, not serving assets from tmp directory etc. We have resolved this by exposing an API in ember-cli via RFC#80. The change has landed in ember-cli@2.12.0-beta.1 and we are in process of making changes to ember-cli-fastboot to use this API. This primarily also lets us deprecate ember fastboot command and allows developers to use ember serve in fastboot environments.

2. Building FastBoot assets: Currently FastBoot builds two sets of assets, one for browser and one for the Fastboot environment. This ends up causing build to run twice to generate the asset. This meeting was primarily to address these painpoints and come up with a plan on how we can address it, unblock and ship FastBoot 1.0.

Background

Today FastBoot provides an addon ember-cli-fastboot that exposes the required fastboot command to build and serve fastboot assets. It assumes a particular file structure in order to build the fastboot assets. For example today fastboot specific implementation lives in app tree under a fastboot folder as follows:

-+ app/
----+ initializers/
--------+ browser/
------------+ foo.js
------------+  ...
--------+ fastboot/
------------+ bar.js
------------+ ...
----+ instance-initializers/
--------+ browser/
------------+ browser-instance.js
------------+  ...
--------+ fastboot/
------------+ bar-instance.js
------------+ ...

FastBoot generates two sets of assets that are filtered based on browser and fastboot trees using private APIs of ember-cli and overriding ember-cli behavior. It also exposes a process environment variable (process.env.EMBER_CLI_FASTBOOT) to let other addons know when the fastboot build is running. In FastBoot, we only load the fastboot version of the asset that contains almost the same browser specific asset and some fastboot overrides. Filtering an entire app tree to create two sets of almost same assets is expensive and results in double builds. It also causes the the app to be built twice and is not ergonomic for developers to iterate building FastBoot apps.

Proposed Solution

In order to solve the double build problem in a performant way, we decided to take a different approach on what and how FastBoot should load the app assets. Instead of creating a different set of asset for FastBoot environment we will generate the browser asset (app.js) as we today and a complimentary asset to the browser asset containing only FastBoot specific overrides. In FastBoot when we will load assets, we will load the browser asset and the fastboot asset. ember-cli-fastboot will only be responsible for generating the complimentary asset using the existing public APIs of ember-cli or its existing downstream dependencies.

In order to generate the complimentary fastboot asset containing only the fastboot overrides, we propose to change the structure of how apps/addons define fastboot overrides. Specifically instead of containing a file structure as: app/instance-initializers/[browser|fastboot]/filename.js addons should define fastboot specific behavior as follows:

-+ app/
----+ initializers/
--------+ foo1.js
--------+  ...
----+ instance-initializers/
--------+ foo.js
--------+  ...
-+ fastboot-app/
----+ initializers/
--------+ foo-fastboot.js
--------+ bar.js

As noted above, we are moving app/[initializers|instance-initializers]/fastboot/*.js to a new directory as fastboot-app. This new direcotry will end up being an additional asset called as appName-fastboot.js containing only fastboot specific overrides.

In FastBoot sandbox, when we load assets we will load the browser (app.js) and this fastboot asset (appName-fastboot.js).

The fastboot-app directory can have the same folder structure as app (example utils, services, routes etc) but it will need to expose fastboot specific overrides in app namespace (just like things from addon namespace are exported in app namespace). fastboot-app AMD modules will have a module id starting with fastboot-app. The reason being we do not want the loader to magically override modules with same id (even though it can do today but that behavior is planned to change in future). An example of this would be something like:

1. Let's say fastboot wants to expose an instance-initializer in FastBoot environment. It will do so by containing a file as fastboot-app/instance-initializers/fastboot.js:

   export default function initialize(instance) {
    // fastboot specific implementation
   }

2. We need to now export this instance initializer in app namespace in order for the app to boot correctly on server side. Therefore, we will define an instance-initializer in app namespace as app/instance-intializer/fastboot.js:

   import require from 'require`; // this corresponds to Ember's loader
   
   export default {
   name: 'fastboot',
   initialize: function(instance) {
      if (FastBoot !== 'undefined') {
        // only require this when in fastboot environment
        require('appName-fastboot/instance-initializers/fastboot')['_default'](instance);
      }
    }

};

 Since this is a shift in how fastboot complaint addons are designed currently, we will **need to add a guide for addons developers to  migrate their fastboot code**. We could also create an automatic migration at build time too in `ember-cli-fastboot`. 

In order for `ember-cli-fastboot` to generate the fastboot asset, we need to do the following:

1. Generate `appName-fastboot.js` asset: 

 In order to do this, `ember-cli-fastboot` will rely on the public API of `ember-cli` to merge the fastboot tree with other assets using `treeForPublic` API. `ember-cli-fastboot` will also be responsible for transpiling `fastboot-app` tree using JS preprocess registeries that are added using `setupPreprocessorRegistry`. At a very highlevel, `ember-cli-fastboot` will use the `treeForPublic` as follows:
 ```javascript
 const BroccoliMergeTrees = require('broccoli-merge-trees');
 const Funnel = require('broccoli-funnel');
 const Concat = require('broccoli-concat');
 const p = require('ember-cli-preprocess-registry/preprocessors');
 
 included(parent) {
    this._appRegistry = parent.registry;
    this._name = parent.name;
 },
 
 treeForPublic(tree) {
     let appName = this._name;
     // TODO: get all addons trees and app
  
     // create a tree of `fastboot-app`
     let extraTree = new Funnel('fastboot-app', {
       destDir: appName + '-fastboot'
     });

     // transpile using the existing added registeries (`preprocessJs` is a public API of `ember-cli-preprocess-registry`)
     var processExtraTree = p.preprocessJs(extraTree, '/', this._name, {
       registry: this._appRegistry
     });

     // TODO: this file needs to be added in fastboot package.json
     // concat and write it to appName-fastboot.js
     var finalTree = Concat(processExtraTree, {
      outputFile: 'assets/' + appName + '-fastboot.js'
     });

     let newTree = new BroccoliMergeTrees([tree, finalTree]);

     return newTree;
 }

When we build, we need to see if we can leverage any existing public APIs of ember-cli or if we need to make an existing private API public via an RFC.

2. Watching fastboot-app files

   We also need to make sure that ember-cli watcher watches files in fastboot-app so that a change in those files can trigger a rebuild.

3. Adding appName-fastboot.js to package.json

   Once the build is done and the asset (appName-fastboot.js) is generated we need to add the asset to fastboot manifest files in package.json. ember-cli-fastboot allows you to provide an array of app and vendor files that will be loaded in the fastboot sandbox context. We need to make sure we add this asset to appFiles after appName.js.

Unresolved items

Following were the unresolved questions that may come in future. Some of the unresolved items are not blockers for FastBoot 1.0 as there are workarounds currently.

1. this.import to generate vendor-fastboot.js

   There are some addons (example ember-network) that require to a different implementation of a vendor file in different environments. Specifically, in ember-network it requires a different implementation for fetch in browser and Node environment. ember-network relies on the process environment (process.env.EMBER_CLI_FASTBOOT) to know when a fastboot build is running and adds the assets. Instead of doing this, in future we want to explore building a complementary fastboot vendor file.

   As a workaround for now, ember-network and other such addons can rely on vendorFiles array in FastBoot manifest and add the Node-fetch implementation in there.

2. Implementing livereload mechanism in FastBoot

   In order for FastBoot to correctly serve assets, it needs to reload the new assets in the sandbox. Browser reload will automatically work once we unify fastboot serving with ember-cli. We need to come up with a proposal on how ember-cli-fastboot should reload the assets in sandbox

   Krati: I have an idea in my mind on how we could do a simple solution that would work in most cases. I'll write it down when I send my PR to unify the serving assets experience.

3. Should we bump FastBoot to 2.0 instead of 1.0?

   Given that with the above proposed changes, addons can no longer will be able to rely on process.env.EMBER_CLI_FASTBOOT this introduces a backward incomaptible change for addons using that environment flag. Technically, we can introduce backward incompatible change since FastBoot is not 1.0 yet, however it may end up being a first bad impression for addon authors that rely on the flag. Ember-Core team will come up with a final consensus.

4. Module unification

   We do not firmly know yet how the proposed solution will work with module unificiation. There are some ideas that could work but we need to come up with a final soluton. Rob/Tom/Stef to talk to Dan and discuss and come up with a final plan.