Towards better alternate runtime environment support (e.g. browser, Bun, React Native, ...)
Focus first on light-client / prover and their dependencies.
A number of node modules are used in both modules or their dependencies. This forces non node consumers to shim/polyfill those methods, which is not ideal. Some of the dependencies ship with polyfill, adding to the confusion.
Unless specified, it should be assumed that lodestar libs do not depend on env specific modules.
Several mitigating strategies can be chosed depending on the node module.
Buffer is used across the codebase. Some usage are required for perfomance reason, some could be replaced by regular js libs.
EventEmitter is used for similar reasons. It is not clear wether the js standard alternative (EventTarget) offers good enough performances.
Several options:
- do not touch critical path
- rely on wrapper functions selecting the right function for the right env (
EventEmittervsEventTarget,BuffervsUInt8Array) - use on the shelf libs (e.g. https://github.com/primus/eventemitter3)
Would be worth identifying current usages and come up with relevant benchmarks.
A dependency is always a security and maintenance risk.
Ideally all dependencies should be audited.
Candidates for removal:
- buffer-xor
- mitt
- eventsource
- bigint-buffer, native support in this fork
- qs
Try to reduce dependencies on fat internal packages with lots of dependencies. Util packages should be as lean as possible.
Candidates for split:
- api
- utils (some utils should ship deps free)
- prover (extract
prover-cli)
CI should be able to identify if modules do not run on targeted envs.
- rely on es-lint rule to prevent usage of NodeJS modules
- test with dumb html page and esm import
- test with bundler known for triggering issues (e.g.
vitest)
- @chainsafe/bls
- @chainsafe/bls-keygen
- bls-eth-wasm
- @chainsafe/persistent-merkle-tree
- @chainsafe/as-sha256
- @noble/hashes
- @chainsafe/ssz
- @chainsafe/as-sha256
- @chainsafe/persistent-merkle-tree
- @lodestar/api
- @chainsafe/persistent-merkle-tree
- @chainsafe/ssz
- @lodestar/config
- @lodestar/params
- @lodestar/types
- @lodestar/utils
- eventsource
- qs
- @lodestar/config
- @chainsafe/ssz
- @lodestar/params
- @lodestar/types
- @lodestar/params
- no deps
- @lodestar/state-transition
- @chainsafe/as-sha256
- @chainsafe/bls
- @chainsafe/blst
- @chainsafe/persistent-merkle-tree
- @chainsafe/persistent-ts
- @chainsafe/ssz
- @lodestar/config
- @lodestar/params
- @lodestar/types
- @lodestar/utils
- bigint-buffer
- buffer-xor
- @lodestar/types
- @chainsafe/ssz
- @lodestar/params
- ethereum-cryptography
- @lodestar/utils
- @chainsafe/as-sha256": "^0.4.1",
- any-signal": "3.0.1",
- bigint-buffer": "^1.1.5",
- case": "^1.6.3",
- chalk": "^5.2.0",
- js-yaml": "^4.1.0"
- mitt
- no deps
- strict-event-emitter-types
- no deps, unmaintained, low-risk (only types)
Transitive deps:
- bls-eth-wasm
- @chainsafe/bls-keygen
- @chainsafe/bls-hd-key
- @noble/hashes
- @scure/bip39$
- @chainsafe/blst
- @types/tar
- node-fetch
- node-gyp
- @chainsafe/persistent-ts
- no deps
- @chainsafe/as-sha256
- no deps
- @noble/hashes
- no deps
- @scure/bip39
- @noble/hashes
- @scure/base
- @scure/base
- no deps
- eventsource
- no deps
- qs
- side-channel
- call-bind
- es-errors
- get-intrinsic
- object-inspect
- each go crazy on deps..
- bigint-buffer
- bindings
- not maintained
- ethereum-cryptography
- no deps
- buffer-xor
- safe-buffer
- no deps, not maintained
I do agree that we should get rid of node specific code and also only use 3rd party dependencies if really necessary.
Ideally, we wanna use APIs that exist in node and the browser but this might either not (yet) be possible or require additional effort to achieve the same behavior as previously, e.g. see custom error handling that was required to properly use native fetch (ChainSafe/lodestar#5811).
And some packages like eventsource we might not be able to replace yet, although I noticed that undici added the API recently in v6.6.2.
Other packages like qs might be hard to replace on the server but we could use a custom implementation on the client as the logic there is rather simple.