Skip to content

Instantly share code, notes, and snippets.

@gilesbradshaw
Last active November 14, 2019 11:39
Show Gist options
  • Star 0 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
  • Save gilesbradshaw/be591e7ac71adaa37b622b2bed5510d4 to your computer and use it in GitHub Desktop.
Save gilesbradshaw/be591e7ac71adaa37b622b2bed5510d4 to your computer and use it in GitHub Desktop.
Bluebird vs native promises for recursion

Native promise leaks memory with recursion

const cdc = ({ x, start }) => {
  if (x % 5000000 === 0) {
    gc()
    console.log({
      used: process.memoryUsage(),
      x,
      time: Date.now() - start,
    })
  }
  return Promise
    .resolve({
      x: x + 1,
      start: x % 5000000 === 0
        ? Date.now()
        : start,
    })
    .then(cdc)
}

cdc({ x: 0, start: Date.now() })

gives..

$ node --expose-gc index.js

{ used:
   { rss: 23244800,
     heapTotal: 6504448,
     heapUsed: 2209560,
     external: 8272 },
  x: 0,
  time: 3 }
{ used:
   { rss: 504827904,
     heapTotal: 496189440,
     heapUsed: 442981968,
     external: 8272 },
  x: 5000000,
  time: 3854 }
{ used:
   { rss: 951590912,
     heapTotal: 942358528,
     heapUsed: 883024184,
     external: 8272 },
  x: 10000000,
  time: 5029 }
{ used:
   { rss: 1383403520,
     heapTotal: 1380663296,
     heapUsed: 1323073856,
     external: 8272 },
  x: 15000000,
  time: 9658 }

<--- Last few GCs --->

[14557:0x103ab2000]    25082 ms: Scavenge 1369.2 (1410.2) -> 1368.6 (1410.7) MB, 11.6 / 0.0 ms  (average mu = 0.175, current mu = 0.134) allocation failure 
[14557:0x103ab2000]    25087 ms: Scavenge 1369.4 (1410.7) -> 1368.8 (1411.2) MB, 3.8 / 0.0 ms  (average mu = 0.175, current mu = 0.134) allocation failure 
[14557:0x103ab2000]    25092 ms: Scavenge 1369.6 (1411.2) -> 1369.0 (1412.2) MB, 4.0 / 0.0 ms  (average mu = 0.175, current mu = 0.134) allocation failure 


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x3ea7bdc4fc7d]
Security context: 0x2242b8b1d949 <JSObject>
    1: cdc [0x2242419038b1] [/Users/giles/test-recurse/index.js:~1] [pc=0x3ea7bdc7db73](this=0x224209185619 <JSGlobal Object>,0x2242da50d831 <Object map = 0x224271fc7da1>)
    2: StubFrame [pc: 0x3ea7bdc3f8ae]
    3: StubFrame [pc: 0x3ea7bdc1c6d2]
    4: EntryFrame [pc: 0x3ea7bdc05c9e]
    5: ExitFrame [pc: 0x3ea7bdc4fc7d]
    6: processTicksAndRejections(ak...

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 1: 0x1000621d5 node::Abort() [/usr/local/bin/node]
 2: 0x100062895 node::OnFatalError(char const*, char const*) [/usr/local/bin/node]
 3: 0x100176777 v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 4: 0x100176718 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [/usr/local/bin/node]
 5: 0x100431730 v8::internal::Heap::UpdateSurvivalStatistics(int) [/usr/local/bin/node]
 6: 0x100433171 v8::internal::Heap::CheckIneffectiveMarkCompact(unsigned long, double) [/usr/local/bin/node]
 7: 0x100430ab7 v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [/usr/local/bin/node]
 8: 0x10042f8a5 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [/usr/local/bin/node]
 9: 0x10042ecf4 v8::internal::Heap::HandleGCRequest() [/usr/local/bin/node]
10: 0x1003feaac v8::internal::StackGuard::HandleInterrupts() [/usr/local/bin/node]
11: 0x1005f4b2e v8::internal::Runtime_StackGuard(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/local/bin/node]
12: 0x3ea7bdc4fc7d 
Abort trap: 6

Bluebird promise no memory leak with recursion

and faster too!

global.Promise = require("Bluebird")

const cdc = ({ x, start }) => {
  if (x % 5000000 === 0) {
    gc()
    console.log({
      used: process.memoryUsage(),
      x,
      time: Date.now() - start,
    })
  }
  return Promise
    .resolve({
      x: x + 1,
      start: x % 5000000 === 0
        ? Date.now()
        : start,
    })
    .then(cdc)
}

cdc({ x: 0, start: Date.now() })

gives

$ node --expose-gc index.js

{ used:
   { rss: 27201536,
     heapTotal: 11223040,
     heapUsed: 3433576,
     external: 8272 },
  x: 0,
  time: 4 }
{ used:
   { rss: 31031296,
     heapTotal: 11747328,
     heapUsed: 3836008,
     external: 8272 },
  x: 5000000,
  time: 1264 }
{ used:
   { rss: 31121408,
     heapTotal: 11747328,
     heapUsed: 3846944,
     external: 8272 },
  x: 10000000,
  time: 1276 }
{ used:
   { rss: 31154176,
     heapTotal: 11747328,
     heapUsed: 3811240,
     external: 8272 },
  x: 15000000,
  time: 1354 }
{ used:
   { rss: 35356672,
     heapTotal: 15941632,
     heapUsed: 3811448,
     external: 8272 },
  x: 20000000,
  time: 1292 }
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment