2022-12-untrusted-code.md

"Untrusted code"

Worst possible approach:

• Run plugins all in the same vm context on server side Bad approach:
• Run plugins in the same vm context on browser side

Ish:

• Run plugins in browser side web workers

Better:

• Run plugins in server side v8 isolates / isolated locked down processes

Engineering Values:

• Standards compliance is very good in our case, since it's going to be easier to break standards than adopt standards later.

  • Many technologies are converging on Web Worker standards (e.g. w3c, deno, and workerd from Cloudflare)
  • Adopting a standard environment increases our chances for portability to change direction in long run. (e.g. switching runtime or security model)
  • More options to emulate in dev environments when the execution environment is standardized.
  • Better compatibility with bundling tooling used for other projects (e.g. Service Worker stuff, deno and workerd can share many tools for web request routing, etc).

• TypeScript is the universal language of the web.

  • Large amounts of tooling and can be made less strict or even accept JavaScript.

• Security is imperative in an environment where we want to enable adopting many different extensions and technologies that interface within AFFiNE.

• Plugins (not Web APIs) empower contributors to extend and improve our platform without needing to maintain their own deployments and infrastructure. Think VS Code extensions over GSuite Apps.

• Documentation empowers us and our contributors to leverage the technology and hire more easily for future growth and feature addition.

Things I've looked at in order of alignment to the mentioned values:

• deno Workers: deno is an alternative to Node.js, and their Workers are a way to run sandboxed code off thread.

  • Positives:
    • Good engineering team and relatively long track record of engineering talent and focus on security
    • TypeScript first
    • Emphasis on standards compliance
    • Rust-friendly ecosystem
    • Relatively strong ecosystem for sandboxable components.
    • High-emphasis on security and granular permissions management
    • Validated by Slack's future platform built on deno.
    • Lots of documentation on moving data into and out of Workers
  • Negatives:
    • Appears to have pretty poor performance overhead
      • From Cole's Macbook Air M1, spinning up N workers showed the following in Activity Monitor:
        • N=1000 => 4.61GB, 1008 threads, 1018 ports
        • N=100 => 483MB, 108 threads, 118 ports
      • This means we probably couldn't just spin every plugin up into it's own worker. We'd probably need to allow workers to spin up and down or even pool them.
  • Maybe negative: Adds a toolset and ecosystem to our mix–I'm not sure if this is a "real" negative, as any sandboxing tool is going to require its own ecosystem that is incompatible with most existing tooling.

• Node.js isolated-vm is an npm package which provides a secure sandbox that runs in separate processes.

  • Positives:
    • Performance/overhead is okay
      • On Cole's Macbook Air M1 spawning N vms:
        • N=1000 => 1.20GB, 20 threads, 664 ports
        • N=100 => 148MB, 20 threads, 69 ports
    • Reasonably secure seeming–several testimonials, and foundational tech (v8 isolates) is used for several high profile server infrastructures (Deno Deploy, Cloudflare Workers/workerd).
  • Negatives:
    • Hard move data into and out of isolated-vm process
  • Questions:
    • Can this compile to Windows/Android? Lots of C++, here.

• Node.js vm2 is an npm package which provides a secure sandbox that runs in the same process.

  • Positives:
    • Low overhead for many isolated scripts to run
      • On Cole's Macbook Air M1 spawning N vms:
        • N=1000 => 255MB, 11 threads, 27 ports
    • Easy to move data into and out of vm2 context
    • Low API surface area to understand (okay-ish documentation is not that big of a deal).
  • Negatives:
    • Has some potential security deficiencies, from README: There are ways to crash the node process from inside the sandbox.
    • The fundamental approach to this makes security a moving target as Node.js/ECMAScript adds functionality. See the following exploit fix, which would not have been necessary if they weren't trying to constantly patch the security vulnerable parts as they come up.
  • Questions I didn't get to:
    • Hard to tell exactly how secure it is (no testimonials) see Snyk, though.
    • Can this compile to Windows/Android? Lots of C++, here.

• Node.js quickjs-emscripten is an npm package and async wrapper over QuickJS loaded via WASM.

  • Positives:
    • Many security fallbacks as a result of it being an isolated VM of QuickJS run in WASM.
    • QuickJS has a testimonial being used at Figma for locally run plugins.
  • Negatives:
    • Very hard to move data into and out of the context–especially with Promises. I experienced similar challenges as the following issue describes
    • It sporadically crashes on some executions with similar errors as the following: [1] 62524 bus error node ./decl-tests.mjs
      • There might be some kind of access pattern that I was doing that led to the crash, but I'd prefer it all to be easier/have a better memory managed API which prevents this kind of issue if it's even something this package could improve.
  • Questions I didn't get to:
    • Can this compile to Windows/Android? Lots of C++, here.

• Cloudflare workerd is a server for deploying 1000s of interconnected workers that can talk to each other with very low overhead.

  • Positives:
    • Emphasis on standards compliance of web workers.
    • Designed by a reputable engineer from Cap'n'Proto.
    • Core tech (v8 isolates approach) validated by being used for Cloudflare Workers.
  • I didn't evaluate this past the point where I found out:
    • This is a fork of a Cloudflare internal Workers product and lacks many features compared to Workers like scaling across multiple processes.
    • This codebase has not been "used in production"
    • This is only compileable to macOS/Linux and unlike some solutions, it is targeted to running on macOS/Linux hardware with little incentive to support a larger variety of targets we might want like for Windows or a Native Android app.