Skip to content

Instantly share code, notes, and snippets.

@dapplion
Created August 26, 2021 09:58
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save dapplion/e079d8faf795f758f1f3b341ee590dc2 to your computer and use it in GitHub Desktop.
Save dapplion/e079d8faf795f758f1f3b341ee590dc2 to your computer and use it in GitHub Desktop.
This code should not OOM, but it does
import {ssz} from "@chainsafe/lodestar-types";
let i = 0;
const heapUsed = process.memoryUsage().heapUsed;
while (true) {
getBigStateTreeBacked();
global.gc();
console.log(i++, (process.memoryUsage().heapUsed - heapUsed) / 1e6, "MB");
}
function getBigStateTreeBacked(): any {
const stateTB = ssz.phase0.BeaconState.defaultTreeBacked();
const validator = ssz.phase0.Validator.defaultValue();
for (let i = 0; i < 250_000; i++) {
stateTB.validators.push(validator);
}
}
// lodestar$ LODESTAR_PRESET=mainnet node --expose-gc ts-node lodestar-ssz-oom.ts
// 0 456.3956 MB
// 1 919.7116 MB
// 2 1388.220224 MB
// 3 1846.017496 MB
// <--- Last few GCs --->
// [26583:0x5ab6bc0] 37176 ms: Mark-sweep (reduce) 2046.8 (2053.8) -> 2045.9 (2053.8) MB, 1838.1 / 0.0 ms (+ 0.0 ms in 29 steps since start of marking, biggest step 0.0 ms, walltime since start of marking 1845 ms) (average mu = 0.150, current mu = 0.016)[26583:0x5ab6bc0] 38694 ms: Mark-sweep (reduce) 2047.0 (2050.8) -> 2046.1 (2052.0) MB, 1515.4 / 0.0 ms (average mu = 0.081, current mu = 0.001) allocation failure scavenge might not succeed
// <--- JS stacktrace --->
// FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
// 1: 0xa25510 node::Abort() [node]
// 2: 0x9664d3 node::FatalError(char const*, char const*) [node]
// 3: 0xb9a8be v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
// 4: 0xb9ac37 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
// 5: 0xd56ca5 [node]
// 6: 0xd5782f [node]
// 7: 0xd6566b v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
// 8: 0xd6922c v8::internal::Heap::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node]
// 9: 0xd3790b v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationType, v8::internal::AllocationOrigin) [node]
// 10: 0x107fbef v8::internal::Runtime_AllocateInYoungGeneration(int, unsigned long*, v8::internal::Isolate*) [node]
// 11: 0x1426919 [node]
// Aborted (core dumped)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment