Skip to content

Instantly share code, notes, and snippets.

@surma surma/README.md
Last active Jul 4, 2020

Embed
What would you like to do?
webpack-emscripten-wasm

Minimal example making webpack and wasm/Emscripten work together.

Build instructions:

  • Clone this gist
  • npm install
  • npm start
  • Open http://localhost:8080
  • Look at console

Note: Docker is required to build this project.

I filed a bug with webpack to make this integration easier.

More questions? Hit me up on Twitter.

License Apache-2.0

<3 Surma

/**
* Copyright 2018 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
#include <emscripten.h>
EMSCRIPTEN_KEEPALIVE
int fib(int n) {
int i, t, a = 0, b = 1;
for (i = 0; i < n; i++) {
t = a + b;
a = b;
b = t;
}
return b;
}
<!doctype html>
<script src="./dist/bundle.js"></script>
/**
* Copyright 2018 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
import fibonacci from './fibonacci.js';
import fibonacciModule from './fibonacci.wasm';
// Since webpack will change the name and potentially the path of the
// `.wasm` file, we have to provide a `locateFile()` hook to redirect
// to the appropriate URL.
// More details: https://kripken.github.io/emscripten-site/docs/api_reference/module.html
const module = fibonacci({
locateFile(path) {
if(path.endsWith('.wasm')) {
return fibonacciModule;
}
return path;
}
});
module.onRuntimeInitialized = () => {
console.log(module._fib(12));
};
{
"name": "lol",
"scripts": {
"build:codec": "docker run --rm -v $(pwd):/src trzeci/emscripten emcc -O3 -s WASM=1 -s EXTRA_EXPORTED_RUNTIME_METHODS='[\"cwrap\"]' -s ALLOW_MEMORY_GROWTH=1 -s MODULARIZE=1 -s 'EXPORT_NAME=\"fibonacci\"' -o ./fibonacci.js fibonacci.c",
"build:bundle": "webpack",
"build": "npm run build:codec && npm run build:bundle",
"serve": "http-server",
"start": "npm run build && npm run serve"
},
"devDependencies": {
"exports-loader": "^0.7.0",
"file-loader": "^1.1.11",
"http-server": "^0.11.1",
"webpack": "^4.8.3",
"webpack-cli": "^2.1.3"
}
}
/**
* Copyright 2018 Google Inc. All Rights Reserved.
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
* http://www.apache.org/licenses/LICENSE-2.0
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
const webpack = require("webpack");
const path = require("path");
module.exports = {
mode: "development",
context: path.resolve(__dirname, "."),
entry: "./index.js",
output: {
path: path.resolve(__dirname, "dist"),
filename: "bundle.js"
},
// This is necessary due to the fact that emscripten puts both Node and web
// code into one file. The node part uses Node’s `fs` module to load the wasm
// file.
// Issue: https://github.com/kripken/emscripten/issues/6542.
browser: {
"fs": false
},
module: {
rules: [
// Emscripten JS files define a global. With `exports-loader` we can
// load these files correctly (provided the global’s name is the same
// as the file name).
{
test: /fibonacci\.js$/,
loader: "exports-loader"
},
// wasm files should not be processed but just be emitted and we want
// to have their public URL.
{
test: /fibonacci\.wasm$/,
type: "javascript/auto",
loader: "file-loader",
options: {
publicPath: "dist/"
}
}
]
},
};
@Quirksmode

This comment has been minimized.

Copy link

Quirksmode commented Sep 7, 2018

Thanks for putting this together, it looks close to what I need to get this working. When I ran 'npm start', I got the following error however:

`Invalid configuration object. Webpack has been initialised using a configuration object that does not match the API schema.

  • configuration has an unknown property 'browser'.`

Any help solving this would be massively appreciated :-)

@ascorbic

This comment has been minimized.

Copy link

ascorbic commented Oct 17, 2018

@Quirksmode: You need to change the browser options to:

node: {
	fs: "empty"
}
@elv-peter

This comment has been minimized.

Copy link

elv-peter commented Nov 14, 2018

$ npm run build:bundle

> lol@ build:bundle /p/e/3rdparty/webpack-emscripten-wasm
> webpack

/p/e/3rdparty/webpack-emscripten-wasm/node_modules/webpack-cli/bin/config-yargs.js:89
				describe: optionsSchema.definitions.output.properties.path.description,
				                                           ^

TypeError: Cannot read property 'properties' of undefined
    at module.exports (/p/e/3rdparty/webpack-emscripten-wasm/node_modules/webpack-cli/bin/config-yargs.js:89:48)
    at /p/e/3rdparty/webpack-emscripten-wasm/node_modules/webpack-cli/bin/webpack.js:60:27
    at Object.<anonymous> (/p/e/3rdparty/webpack-emscripten-wasm/node_modules/webpack-cli/bin/webpack.js:515:3)
    at Module._compile (internal/modules/cjs/loader.js:707:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:718:10)
    at Module.load (internal/modules/cjs/loader.js:605:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:544:12)
    at Function.Module._load (internal/modules/cjs/loader.js:536:3)
    at Module.require (internal/modules/cjs/loader.js:643:17)
    at require (internal/modules/cjs/helpers.js:22:18)
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! lol@ build:bundle: `webpack`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the lol@ build:bundle script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.
@elv-peter

This comment has been minimized.

Copy link

elv-peter commented Nov 14, 2018

It worked after the configuration change above and updating webpack-cli to 3.1.2

And "exports-loader" doesn't seem to be necessary. I think it also doesn't do anything if you inspect (e.g. by adding console.log to) the loader output.

Also, Emscripten has a "-s ENVIRONMENT=web" option

@shayc

This comment has been minimized.

Copy link

shayc commented Nov 26, 2018

@elv-peter thank you, I've been suffering so much trying to integrate. Managed to build with your help! May you win a billion internet points.

@kevinchar93

This comment has been minimized.

Copy link

kevinchar93 commented Nov 28, 2018

Stumbled upon this thread while looking for answers, appears that there has now been change made to Emscripten to stop it outputting node.js related code in any output .js file.

Adding -s ENVIRONMENT="web" to my build line resolved the issues I was seeing.
Hope thins helps!

See PR emscripten-core/emscripten#6565

@kevinchar93

This comment has been minimized.

Copy link

kevinchar93 commented Mar 29, 2019

Came back to add another update, used this technique to successfully load a .wasm file from a location other than the same directory as the .js file that Emscripten generates to load said .wasm file.

My issues was effectively the same as this question:
https://stackoverflow.com/questions/46332699/how-can-i-load-wasm-files-stored-in-a-subdirectory

Some differences I experienced / gotchas:

  • if the publicPath for the file-loader config is incorrect fetch requests for the .wasm file will fail (404) causing the streaming compilation to fail (expected magic word 00 61 73 6d) as it tired to load the response, so double check that this is correct.
  • I didn't have to use the exports-loader , not sure if this will cause issues later, hopefully not!

Many Thanks!

@GreLI

This comment has been minimized.

Copy link

GreLI commented Jun 7, 2019

Compiling with option -s "ENVIRONMENT='web'" solves the fs problem. (There is also, FILESYSTEM=0 option.)

@theochampion

This comment has been minimized.

Copy link

theochampion commented Jan 7, 2020

Hey guys,

I'm trying to load a WASM module from a react project created with create-react-app and I'm having trouble finding the correct publicPath cause I keep getting

wasm streaming compile failed: CompileError: WebAssembly.instantiateStreaming(): expected magic word 00 61 73 6d, found 6d 6f 64 75 @+0

my glue code and my wasm sources are located in /src/video/ so I have the following webpack rule:

 // override default create-react-app config
  config.module.rules.push({
    test: /video\/rPPG_asm\.wasm$/,
    type: 'javascript/auto',
    loader: 'file-loader',
    options: {
      publicPath: 'public/'
    }
  });

When I look at the chrome network console however, a call is made by my glue code at

http://localhost:3000/static/media/rPPG_asm.9df80b34.wasm

Any Ideas ?

@theochampion

This comment has been minimized.

Copy link

theochampion commented Jan 27, 2020

Following my previous comment, I digged in a bit more:

First, I think it's important to mention that I'm working with a webpack config created by create-react-app that I override using https://github.com/timarney/react-app-rewired

With this webpack override config

  config.module.rules.push({
    test: /rPPG_asm\.wasm$/,
    type: 'javascript/auto', // ← !!
    loader: 'file-loader',
    options: {
      publicPath: 'public/static/wasm/'
    }
  });

I get two files are generated in the build:

/public/static/media/rPPG_asm.6074f4a1.wasm which contains

module.exports = "public/static/wasm/dad54818a8426f637a10e1e56c6659b5.wasm";

and /public/static/wasm/dad54818a8426f637a10e1e56c6659b5.wasm

which contains the actual wasm

Thing is, the path provided in the LocateFile function is this one /public/static/media/rPPG_asm.6074f4a1.wasm so the glue code tries to load that as valid WASM hence the magic number error.

What am I missing here ? :)

@skandonkumar

This comment has been minimized.

Copy link

skandonkumar commented Mar 4, 2020

EMScripten locateFile() not reading .WASM file after bundling with WebPack #10637

I have converted my C library to WASM and wrapper JS using EMScripten. I also wrote my own JS library. I want to integrate both the libraries. I used Webpack to bundle my JS library including my Emscripten wrapper JS and added the WASM file using "file-loader" of WebPack. Initially, the Emscripten generated JS wrapper reads the WASM file using locateFile() method. But after bundling, the locate file is unable to locate the WASM file. Can someone please help me?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.