Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
Loading ES Modules in Electron 2.0.0 using Protocol

Now that Electron has hit Chrome's 60s we can have proper module support. But of course, the standard (for very good reasons) prevents loading modules from file: and it makes sense for both Electron and NW.js to adhere to the statusquo.

So if you were really excited and this bums you, don't worry, you are in for awesome things.

The future Electron eco-system offers two options for native ES modules:

  1. Custom Electron protocol via Chromium's module loading subsystem

  2. Custom NodeJS loaders via Node's module loading subsystem recommended

Both methods can coexist. In fact, so far from tests, it seems that at least for hybrid applications, using both methods together will be the more suitable path.

This focuses on the first.

Revision: Draft 2

const { mainModule } = process, { error } = console;
function createProtocol(scheme, base, normalize = true) {
const mimeTypeFor = require('./mime-types'),
{ app, protocol } = require('electron'),
{ URL } = require('url'),
{ readFileSync: read } = require('fs'),
{ _resolveFilename: resolve } = require('module');
// Should only be called after app:ready fires
if (!app.isReady())
return app.on('ready', () => createProtocol(...arguments));
// Normalize standard URLs to match file protocol format
normalize = !normalize
? url => new URL(url).pathname
: url => new URL(
url.replace(/^.*?:[/]*/, `file:///`) // `${scheme}://./`
).pathname.replace(/[/]$/, '');
protocol.registerBufferProtocol(
scheme,
(request, respond) => {
let pathname, filename, data, mimeType;
try {
// Get normalized pathname from url
pathname = normalize(request.url);
// Resolve absolute filepath relative to mainModule
filename = resolve(`.${pathname}`, mainModule);
// Read contents into a buffer
data = read(filename);
// Resolve mimeType from extension
mimeType = mimeTypeFor(filename);
// Respond with mimeType & data
respond({ mimeType, data });
} catch (exception) {
error(exception, { request, pathname, filename, data, mimeType });
}
},
(exception) =>
exception && error(`Failed to register ${scheme} protocol`, exception)
);
}
module.exports = createProtocol;
<!DOCTYPE html>
<html>
<head>
<!-- Needed if not loading page from app://./index.html -->
<base href="app://./" />
<script type="module" src="app:test-module.mjs"></script>
</head>
<body>
Check the console!
</body>
</html>
const { app, protocol } = require('electron');
// Base path used to resolve modules
const base = app.getAppPath();
// Protocol will be "app://./…"
const scheme = 'app';
{ /* Protocol */
// Registering must be done before app::ready fires
// (Optional) Technically not a standard scheme but works as needed
protocol.registerStandardSchemes([scheme], { secure: true });
// Create protocol
require('./create-protocol')(scheme, base);
}
{ /* BrowserWindow */
let browserWindow;
const createWindow = () => {
if (browserWindow) return;
browserWindow = new BrowserWindow();
// Option A — using the custom protocol
// browserWindow.loadURL('app://./index.html');
// Option B — directly from file
browserWindow.loadFile('index.html');
}
app.isReady()
? createWindow()
: app.on('ready', createWindow);
}
const { extname } = require('path');
const mime = filename =>
mime[extname(`${filename || ''}`).toLowerCase()];
mime[''] = 'text/plain',
mime['.js'] =
mime['.ts'] =
mime['.mjs'] = 'text/javascript',
mime['.html'] =
mime['.htm'] = 'text/html',
mime['.json'] = 'application/json',
mime['.css'] = 'text/css',
mime['.svg'] = 'application/svg+xml';
module.exports = mime;
const message = 'Hello World!';
console.trace(message);
export default message;
@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 6, 2018

app.isReady should probably be replaced with app.isReady(). I would also replace readFileSync()with readFile() to keep things fast.

Still, the demo app does not seem to be working for me, i.e. a blank page is loaded instead of index.html.

@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 6, 2018

I got it working with some tweaks: https://gist.github.com/jarek-foksa/0f6e82bdaf8fb1962c9e14035a8725e4

I'm still trying to figure out what should be the correct content security policy meta tag. <meta http-equiv="Content-Security-Policy" content="script-src 'self'; object-src 'self';"> does not allow resources with app:// protocol.

@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 6, 2018

Adding the code below to the <head> seems to resolve the content security policy warnings:

<meta http-equiv="Content-Security-Policy" content="script-src 'self' app://*; object-src 'self' app://*;">

This also seems to work:

<meta http-equiv="Content-Security-Policy" content="script-src 'self' app:; object-src 'self' app:;">
@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 6, 2018

@SMotaal

This comment has been minimized.

Copy link
Owner Author

SMotaal commented Mar 6, 2018

Sorry, test project's setup is very different so I was literally gisting in the browser, but I still want to fix the cause (and will fix app.isReay)!

Now trying to pin point from your changes what was causing the blank page, but I realize that changed all of 😉 can you nail the exact cause of the blank page down?

I would also replace readFileSync()with readFile() to keep things fast

Not necessarily, since the app is simply loading "on-demand" to the single client (not online) it is actually far more efficient to get things done immediately. If you plan to load huge files, you are better off using a stream protocol (not a buffer one). What would really make it more performant is proper caching for certain high-demand assets, but that is secondary to the concept (and if not done right not done is much more reliable) not to mention there is likely a flag that that will let chromium do that anyway.

@SMotaal

This comment has been minimized.

Copy link
Owner Author

SMotaal commented Mar 7, 2018

I'm still trying to figure out what should be the correct content security policy meta tag. does not allow resources with app:// protocol.

I am using the following web preferences and not getting CSP related warnings for the app: protocol:

{
    webPreferences: {
      allowRunningInsecureContent: false,
      experimentalFeatures: true,
    }
}

But if you are talking about this:
screenshot 2018-03-06 18 59 19

Check this out: electron/electron#12035

You can add the following to your main.js to mute those out (but they only show in development anyway):

process.env['ELECTRON_DISABLE_SECURITY_WARNINGS'] = true;
@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 7, 2018

I think I was getting a blank page previously because I didn't change app.isReady to app.isReady() in both create-protocol.js and main.js.

I'm not a Node.js expert, but doesn't fs.readFileSync() lock the whole main process so that it can't e.g. respond to other requests from the renderer process until the file is read from disk?

If the goal is to add this example to the official Electron documentation, then I think disabling the security warnings is not the best advice.

@SMotaal

This comment has been minimized.

Copy link
Owner Author

SMotaal commented Mar 8, 2018

Well, though flattering, I am not sure this will be "Electron documentation" material, I'm just a user with like three weeks worth of test with Electron (since beta 2.0.0-beta.1) and maybe twice that time in past eras.

I'm not a Node.js expert, but doesn't fs.readFileSync() lock the whole main process so that it can't e.g. respond to other requests from the renderer process until the file is read from disk?

So in this particular instance, I was torn between conventional wisdom to always IO asynchronously and between how Node's own module system handles file reading. They basically use a special flavour of fs exposed to them through process.binding('fs') which provides synchronous internalModuleReadFile and internalModuleStat, which I often rely on in my module loading tests to limit the number of variables in terms of performance. This internal read method simply returns contents as a string or an empty string if the operation fails (ie file not found or no access). Since we need the contents in a buffer, I assumed the internal read was basically reading a buffer, converting to string, and then we would be creating yet another buffer from that string to pass it to Electron, then chromium would convert to string… etc.

So essentially, readFileSync sounded like the closest possible option to internal read file, but without redundant buffering.

I would always async reading, unless it becomes counterproductive, and the folks that designed fs.readFile thought it not to be their weapon of choice in very few instances (or maybe even for that very specific instance) and I have not encountered evidence that would support the implications of asynchronous loading resources that are most likely being requested sequential for optimal performance.

So, the one thing I would explore would be to use readFile for things that if not delivered instantaneously would block rendering or delay either "first-paint" or "interactive", and then use readFileSync for anything huge and anything that does not impact the "feels-like-native" experience. And for really huge though, I would suppose either bypassing protocols altogether or using a stream protocol as appropriate.

@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 8, 2018

I have found one more issue - SVG files would fail to load until I changed the mime type from "application/svg+xml" to "image/svg+xml". I'm still unable to get SVG <use> elements to work though.

@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 8, 2018

Another odd issue is that modules would fail to load with "net::ERR_UNKNOWN_URL_SCHEME" error if I create a browser window with a persistent partition.

@jarek-foksa

This comment has been minimized.

Copy link

jarek-foksa commented Mar 8, 2018

Also, I had to use webFrame.registerURLSchemeAsPrivileged() in order to make window.fetch() work with app: URLs (I have updated my gist).

@Ontopic

This comment has been minimized.

Copy link

Ontopic commented May 4, 2018

After building Electron with the upgrade-to-chromium-66 branch I'm able to use native import on browser and renderer side.

const Loader = process.NativeModule.require('internal/process/esm_loader')

Loader.setup()

Loader.loaderPromise.then(loader => {
  loader.import('file:///x/main.mjs')
})

After that import can be used in main.mjs (or renderer.mjs if you use it as a preload script).

No prefix needed in the import specifier, even for node_modules (think import fs from 'fs' or import _ from 'lodash'). Importing Electron itself does not work yet, but that's literally the only require I need.

@Ontopic

This comment has been minimized.

Copy link

Ontopic commented May 4, 2018

@SMotaal Thought I would share, since your sharing so far has been really worthwhile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.