Skip to content

Instantly share code, notes, and snippets.

@SMotaal
Last active January 13, 2023 20:07
Show Gist options
  • Star 21 You must be signed in to star a gist
  • Fork 2 You must be signed in to fork a gist
  • Save SMotaal/f1e6dbb5c0420bfd585874bd29f11c43 to your computer and use it in GitHub Desktop.
Save SMotaal/f1e6dbb5c0420bfd585874bd29f11c43 to your computer and use it in GitHub Desktop.
Loading ES Modules in Electron 2.0.0 using Protocol

Now that Electron has hit Chrome's 60s we can have proper module support. But of course, the standard (for very good reasons) prevents loading modules from file: and it makes sense for both Electron and NW.js to adhere to the statusquo.

So if you were really excited and this bums you, don't worry, you are in for awesome things.

The future Electron eco-system offers two options for native ES modules:

  1. Custom Electron protocol via Chromium's module loading subsystem

  2. Custom NodeJS loaders via Node's module loading subsystem recommended

Both methods can coexist. In fact, so far from tests, it seems that at least for hybrid applications, using both methods together will be the more suitable path.

This focuses on the first.

Revision: Draft 2

const { mainModule } = process, { error } = console;
function createProtocol(scheme, base, normalize = true) {
const mimeTypeFor = require('./mime-types'),
{ app, protocol } = require('electron'),
{ URL } = require('url'),
{ readFileSync: read } = require('fs'),
{ _resolveFilename: resolve } = require('module');
// Should only be called after app:ready fires
if (!app.isReady())
return app.on('ready', () => createProtocol(...arguments));
// Normalize standard URLs to match file protocol format
normalize = !normalize
? url => new URL(url).pathname
: url => new URL(
url.replace(/^.*?:[/]*/, `file:///`) // `${scheme}://./`
).pathname.replace(/[/]$/, '');
protocol.registerBufferProtocol(
scheme,
(request, respond) => {
let pathname, filename, data, mimeType;
try {
// Get normalized pathname from url
pathname = normalize(request.url);
// Resolve absolute filepath relative to mainModule
filename = resolve(`.${pathname}`, mainModule);
// Read contents into a buffer
data = read(filename);
// Resolve mimeType from extension
mimeType = mimeTypeFor(filename);
// Respond with mimeType & data
respond({ mimeType, data });
} catch (exception) {
error(exception, { request, pathname, filename, data, mimeType });
}
},
(exception) =>
exception && error(`Failed to register ${scheme} protocol`, exception)
);
}
module.exports = createProtocol;
<!DOCTYPE html>
<html>
<head>
<!-- Needed if not loading page from app://./index.html -->
<base href="app://./" />
<script type="module" src="app:test-module.mjs"></script>
</head>
<body>
Check the console!
</body>
</html>
const { app, protocol } = require('electron');
// Base path used to resolve modules
const base = app.getAppPath();
// Protocol will be "app://./…"
const scheme = 'app';
{ /* Protocol */
// Registering must be done before app::ready fires
// (Optional) Technically not a standard scheme but works as needed
protocol.registerStandardSchemes([scheme], { secure: true });
// Create protocol
require('./create-protocol')(scheme, base);
}
{ /* BrowserWindow */
let browserWindow;
const createWindow = () => {
if (browserWindow) return;
browserWindow = new BrowserWindow();
// Option A — using the custom protocol
// browserWindow.loadURL('app://./index.html');
// Option B — directly from file
browserWindow.loadFile('index.html');
}
app.isReady()
? createWindow()
: app.on('ready', createWindow);
}
const { extname } = require('path');
const mime = filename =>
mime[extname(`${filename || ''}`).toLowerCase()];
mime[''] = 'text/plain',
mime['.js'] =
mime['.ts'] =
mime['.mjs'] = 'text/javascript',
mime['.html'] =
mime['.htm'] = 'text/html',
mime['.json'] = 'application/json',
mime['.css'] = 'text/css',
mime['.svg'] = 'application/svg+xml';
module.exports = mime;
const message = 'Hello World!';
console.trace(message);
export default message;
@jarek-foksa
Copy link

app.isReady should probably be replaced with app.isReady(). I would also replace readFileSync()with readFile() to keep things fast.

Still, the demo app does not seem to be working for me, i.e. a blank page is loaded instead of index.html.

@jarek-foksa
Copy link

jarek-foksa commented Mar 6, 2018

I got it working with some tweaks: https://gist.github.com/jarek-foksa/0f6e82bdaf8fb1962c9e14035a8725e4

I'm still trying to figure out what should be the correct content security policy meta tag. <meta http-equiv="Content-Security-Policy" content="script-src 'self'; object-src 'self';"> does not allow resources with app:// protocol.

@jarek-foksa
Copy link

jarek-foksa commented Mar 6, 2018

Adding the code below to the <head> seems to resolve the content security policy warnings:

<meta http-equiv="Content-Security-Policy" content="script-src 'self' app://*; object-src 'self' app://*;">

This also seems to work:

<meta http-equiv="Content-Security-Policy" content="script-src 'self' app:; object-src 'self' app:;">

@jarek-foksa
Copy link

@SMotaal
Copy link
Author

SMotaal commented Mar 6, 2018

Sorry, test project's setup is very different so I was literally gisting in the browser, but I still want to fix the cause (and will fix app.isReay)!

Now trying to pin point from your changes what was causing the blank page, but I realize that changed all of 😉 can you nail the exact cause of the blank page down?

I would also replace readFileSync()with readFile() to keep things fast

Not necessarily, since the app is simply loading "on-demand" to the single client (not online) it is actually far more efficient to get things done immediately. If you plan to load huge files, you are better off using a stream protocol (not a buffer one). What would really make it more performant is proper caching for certain high-demand assets, but that is secondary to the concept (and if not done right not done is much more reliable) not to mention there is likely a flag that that will let chromium do that anyway.

@SMotaal
Copy link
Author

SMotaal commented Mar 7, 2018

I'm still trying to figure out what should be the correct content security policy meta tag. does not allow resources with app:// protocol.

I am using the following web preferences and not getting CSP related warnings for the app: protocol:

{
    webPreferences: {
      allowRunningInsecureContent: false,
      experimentalFeatures: true,
    }
}

But if you are talking about this:
screenshot 2018-03-06 18 59 19

Check this out: electron/electron#12035

You can add the following to your main.js to mute those out (but they only show in development anyway):

process.env['ELECTRON_DISABLE_SECURITY_WARNINGS'] = true;

@jarek-foksa
Copy link

I think I was getting a blank page previously because I didn't change app.isReady to app.isReady() in both create-protocol.js and main.js.

I'm not a Node.js expert, but doesn't fs.readFileSync() lock the whole main process so that it can't e.g. respond to other requests from the renderer process until the file is read from disk?

If the goal is to add this example to the official Electron documentation, then I think disabling the security warnings is not the best advice.

@SMotaal
Copy link
Author

SMotaal commented Mar 8, 2018

Well, though flattering, I am not sure this will be "Electron documentation" material, I'm just a user with like three weeks worth of test with Electron (since beta 2.0.0-beta.1) and maybe twice that time in past eras.

I'm not a Node.js expert, but doesn't fs.readFileSync() lock the whole main process so that it can't e.g. respond to other requests from the renderer process until the file is read from disk?

So in this particular instance, I was torn between conventional wisdom to always IO asynchronously and between how Node's own module system handles file reading. They basically use a special flavour of fs exposed to them through process.binding('fs') which provides synchronous internalModuleReadFile and internalModuleStat, which I often rely on in my module loading tests to limit the number of variables in terms of performance. This internal read method simply returns contents as a string or an empty string if the operation fails (ie file not found or no access). Since we need the contents in a buffer, I assumed the internal read was basically reading a buffer, converting to string, and then we would be creating yet another buffer from that string to pass it to Electron, then chromium would convert to string… etc.

So essentially, readFileSync sounded like the closest possible option to internal read file, but without redundant buffering.

I would always async reading, unless it becomes counterproductive, and the folks that designed fs.readFile thought it not to be their weapon of choice in very few instances (or maybe even for that very specific instance) and I have not encountered evidence that would support the implications of asynchronous loading resources that are most likely being requested sequential for optimal performance.

So, the one thing I would explore would be to use readFile for things that if not delivered instantaneously would block rendering or delay either "first-paint" or "interactive", and then use readFileSync for anything huge and anything that does not impact the "feels-like-native" experience. And for really huge though, I would suppose either bypassing protocols altogether or using a stream protocol as appropriate.

@jarek-foksa
Copy link

jarek-foksa commented Mar 8, 2018

I have found one more issue - SVG files would fail to load until I changed the mime type from "application/svg+xml" to "image/svg+xml". I'm still unable to get SVG <use> elements to work though.

@jarek-foksa
Copy link

Another odd issue is that modules would fail to load with "net::ERR_UNKNOWN_URL_SCHEME" error if I create a browser window with a persistent partition.

@jarek-foksa
Copy link

Also, I had to use webFrame.registerURLSchemeAsPrivileged() in order to make window.fetch() work with app: URLs (I have updated my gist).

@Ontopic
Copy link

Ontopic commented May 4, 2018

After building Electron with the upgrade-to-chromium-66 branch I'm able to use native import on browser and renderer side.

const Loader = process.NativeModule.require('internal/process/esm_loader')

Loader.setup()

Loader.loaderPromise.then(loader => {
  loader.import('file:///x/main.mjs')
})

After that import can be used in main.mjs (or renderer.mjs if you use it as a preload script).

No prefix needed in the import specifier, even for node_modules (think import fs from 'fs' or import _ from 'lodash'). Importing Electron itself does not work yet, but that's literally the only require I need.

@Ontopic
Copy link

Ontopic commented May 4, 2018

@SMotaal Thought I would share, since your sharing so far has been really worthwhile

@bobjunga
Copy link

I am trying to figure out the best way to write a new electron app in 2023 using only ES modules. Is this still a good approach? Or are there better ways now? I do not want to use a local server to deliver esm.

Do you know if its possible at this time to enable the file: protocol to work in the renderer? I am a bit confused about why enabling nodeIntegration does not also allow the ESM loader to load from file:

BTW, my goal is to disable network fetch by default and treat security from a local perspective. If my electron app does not allow loading remote URLs, I am thinking that is would be better to allow the file: protocol to work from the renderer so that I dont have to use a custom protocol schema in my imports.

@SMotaal
Copy link
Author

SMotaal commented Jan 12, 2023

@SMotaal Thought I would share, since your sharing so far has been really worthwhile

@Ontopic been off grid, but just noticed this and it is much appreciated 👍

@SMotaal
Copy link
Author

SMotaal commented Jan 12, 2023

Do you know if its possible at this time to enable the file: protocol to work in the renderer?

@bobjunga ah, been a long time, but per my past understanding, the approach is similar to @jarek-foksa's comments from before which was through registerFileProtocol (or something else maybe) and registerSchemeAsPrivileged but following current examples.

This has to do with browser layers for security on standard schemes, which likely still applies today, but I could be mistaken.

That's as much as I am able at this point to trust would likely work, but I have not worked with electron since 2019.

Hope this helps :)

@bobjunga
Copy link

@SMotaal thanks for the quick reply. After pouring over chromium,blink, and V8 all morning, its dawning on me why I will not be able to to ES modules in electron's renderer as I would like. I will stick to using esm for almost modern syntax without actually using the builtin ES module support.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment