Skip to content

Instantly share code, notes, and snippets.

@xaliphostes
Last active February 7, 2024 08:53
Show Gist options
  • Save xaliphostes/6869989fadfb055ea40c2987b1150549 to your computer and use it in GitHub Desktop.
Save xaliphostes/6869989fadfb055ea40c2987b1150549 to your computer and use it in GitHub Desktop.
Emscripten pthread lib in web worker

A small multi-threaded C++ library (so, does not include any main) compiled with emscripten and pthread running under the main thread in the browser, a web worker or using node.js.

Note: Emscripten will produce lib.js.

Using a web browser

Running this lib in the main thread is fine but, obviously, block the main thread. Running it using a web worker leads no error with modification of the worker script (see below)

Using node.js

Run fine except that it prevents Node app from ever exiting.

#!/bin/bash
emcc -std=c++17 \
-O3 --bind \
-s SINGLE_FILE=1 \
-s PTHREAD_POOL_SIZE=4 \
-s USE_PTHREADS=1 \
-s ASSERTIONS=1 \
-s EXPORT_NAME=LibModule \
-s WASM=1 \
-s MODULARIZE=1 \
-s ALLOW_MEMORY_GROWTH=1 \
-o lib.js \
main.cxx
<!DOCTYPE html>
<html>
<title>pthread test</title>
<head>
<style>
.styled:hover {
background-color: rgb(140, 139, 139);
}
</style>
</head>
<body>
<button class="favorite styled" type="button" onclick="runMain('thread')">Main thread</button>
<button class="favorite styled" type="button" onclick="runMain('serial')">Main serial</button><br><br>
<button class="favorite styled" type="button" onclick="runWorker('thread')">Worker thread</button>
<button class="favorite styled" type="button" onclick="runWorker('serial')">Worker Serial</button>
<script src="lib.js"></script>
<script>
function runWorker(method) {
const worker = new Worker('./worker.js')
worker.onmessage = e => console.log(e)
worker.postMessage(method)
}
function runMain(method) {
LibModule().then( lib => {
const a = performance.now()
method === 'thread' ? lib.thread() : lib.serial() ;
console.log( `${method} done in ${(performance.now()-a)}ms` )
})
}
</script>
</body>
</html>
#include <string>
#include <thread>
#include <future>
#include <iostream>
#include <vector>
#include <emscripten/bind.h>
using namespace emscripten ;
std::vector<int> s = {40000, 40000, 40000, 40000} ;
void longCompute(int id, int iterations) {
double val = 0 ;
for (int i=0; i<10000; ++i) {
for (int j=0; j<iterations; ++j) {
val += 3.1415 ;
}
}
std::cout << "thread " << (id) << ": " << val << std::endl ;
}
void testThread() {
std::cout << "Using std::thread" << std::endl ;
std::vector<std::thread> threads ;
for (int i = 0; i < s.size(); i++) {
threads.push_back( std::thread(longCompute, i, s[i]) ) ;
}
std::cerr << "start threads"<< std::endl ;
for (auto &th : threads) {
th.join() ;
}
std::cerr << "end threads"<< std::endl ;
}
void testSerial() {
std::cout << "Using serial" << std::endl ;
for (int i = 0; i < s.size(); i++) {
longCompute(i, s[i]) ;
}
std::cerr << "end serial"<< std::endl ;
}
// ------------------------------
// Only expose 2 functions in js
// ------------------------------
EMSCRIPTEN_BINDINGS(test) {
function("thread", &testThread) ;
function("serial", &testSerial) ;
}
#!/bin/bash
# Make sure node version is greater than 15.1.0
ver="$(node --version)"
ver=${ver:1} # remove the prefix 'v'
major="$(cut -d '.' -f 1 <<< "$ver")"
minor="$(cut -d '.' -f 2 <<< "$ver")"
if [ $major -ge 15 ] && [ $minor -ge 1 ]; then
node --experimental-wasm-threads script.js
else
echo "Cannot run node. Require version 15.1.0 or greater"
fi
// ----------------------------------------
// To be used with node.js
// ----------------------------------------
const LibModule = require('./lib')
function runMain(method) {
LibModule().then( lib => {
const a = performance.now()
method === 'thread' ? lib.thread() : lib.serial() ;
console.log( `${method} done in ${(performance.now()-a)}ms` )
})
}
runMain('thread')
runMain('serial')
// ----------------------------------------
// Used with index.html
// ----------------------------------------
importScripts("lib.js")
onmessage = e => {
const Lib = {
locateFile: (file) => file,
onRuntimeInitialized: () => {
const a = performance.now()
e.data === 'thread' ? Lib.thread() : Lib.serial() ;
console.log( `${e.data} done in ${(performance.now()-a)}ms` )
},
mainScriptUrlOrBlob: "lib.js",
};
// LibModule is the name of the exported library with emscripten
LibModule(Lib)
}
// Was previously (loading of the worker not working)
/*
importScripts("lib.js")
onmessage = e => {
LibModule().then( lib => {
const a = performance.now()
e.data === 'thread' ? lib.thread() : lib.serial() ;
console.log( `${e.data} done in ${(performance.now()-a)}ms` )
})
}
*/
@Alexufo
Copy link

Alexufo commented Jul 10, 2021

at 28 line
https://gist.github.com/xaliphostes/6869989fadfb055ea40c2987b1150549#file-worker-js-L28

Did you try to put Lib inside LibModule(Lib).then( lib => { ?

@ravisumit33
Copy link

Why have you used "lib.js" both in index.html & worker.js in case of a web worker?

@ravisumit33
Copy link

Also can you please explain the use of mainScriptUrlOrBlob ?

@Alexufo
Copy link

Alexufo commented Jul 24, 2021

Why have you used "lib.js" both in index.html & worker.js in case of a web worker?

It is a browser limitations. Browser can create a thread only from js file and we must provide path to it.
mainScriptUrlOrBlob - main script.
Better explanation
https://web.dev/webassembly-threads/

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment