V8 Promise Benchmarks

README.md

V8 Promise Benchmarks

EDIT: 2021-01-10
Fixed here: nodejs/node#34291

EDIT: 2020-07-10
Explanation here: nodejs/node#33384 (comment)

By benchmarking promises against different version of Node.js, I noticed a performance drop in traditional promises (then/catch) between Node.js 10 and Node.js 12.

With further investigation I'm now able to say that the performance drop occur between v8 7.4 (Node.js 12.0.0) and v8 7.5 (Node.js 12.5.0).

The performance drop is still present in recent version of v8 (tested with v8 8.1 on Node.js 14.2.0)

Promises are between 3x and 5x slower in v8 7.5 than in v8 7.4 and they are still 2x slower in v8 8.1

You can run the benchmarks by yourself by executing bash run-benchmark.sh. Those benchmark have been realized on a Scaleway GP1-XS instance.

With 50k promises:

	Node.js 12.0.0 (v8 7.4)	Node.js 12.5.0 (v8 7.5)	Node.js 14.2.0 (v8 8.1)	node-v15.0.0 (v8 8.3)
Promise Native	5482	10819	10203	9509
Async Await	4237	8682	5934	7891

With 100k promises:

	Node.js 12.0.0 (v8 7.4)	Node.js 12.5.0 (v8 7.5)	Node.js 14.2.0 (v8 8.1)	node-v15.0.0 (v8 8.3)
Promise Native	13616	37721	27043	30658
Async Await	10150	19357	11529	22605

See the complete data on Google Sheet

run-benchmark.sh

#!/bin/bash

node_arch=${NODE_ARCH:-linux-x64}
node_versions="12.0.0 12.5.0 14.2.0"

echo "Downloading Node.js $node_versions for $node_arch (re-run with NODE_ARCH=darwin-x64 for macos)..."

mkdir nodes/

cd nodes/

for node_version in $node_versions;
do
  wget https://nodejs.org/dist/v$node_version/node-v$node_version-$node_arch.tar.xz
  tar xf node-v$node_version-$node_arch.tar.xz
done

wget https://nodejs.org/download/nightly/v15.0.0-nightly202005135bb4d01fbe/node-v15.0.0-nightly202005135bb4d01fbe-$node_arch.tar.xz
tar xf node-v15.0.0-nightly202005135bb4d01fbe-$node_arch.tar.xz

cd ../

echo "Cloning benchmarks..."

git clone --depth=1 https://github.com/petkaantonov/bluebird

cd bluebird/benchmark

export NODE_ENV=production

npm install

benchmark_files="./doxbee-sequential/promises-native-async-await.js ./doxbee-sequential/promises-ecmascript6-native.js"
benchmark_runner="performance.js"

# First benchmark with few promises and less presure on memory
promises=50000
echo "Benchmarking with $promises promises"

node_versions_and_nightly="$node_versions 15.0.0-nightly202005135bb4d01fbe"

for node_version in $node_versions_and_nightly;
do
  node_binary=../../nodes/node-v$node_version-$node_arch/bin/node
  echo $node_binary
  v8_version=$($node_binary -p process.versions.v8)

  echo "Benchmark Node.js v$node_version with and v8 $v8_version"

  echo $benchmark_files | sed -e 's|\.js||' | xargs $node_binary $benchmark_runner --p 1 --t 1 --n $promises
done

# Then benchmark with twice more promises
promises=100000
echo "Benchmarking with $promises promises"

for node_version in $node_versions_and_nightly;
do
  node_binary=../../nodes/node-v$node_version-$node_arch/bin/node
  v8_version=$($node_binary -p process.versions.v8)

  echo "Benchmark Node.js v$node_version with and v8 $v8_version"

  echo $benchmark_files | sed -e 's|\.js||' | xargs $node_binary $benchmark_runner --p 1 --t 1 --n $promises
done