Skip to content

Instantly share code, notes, and snippets.

@piglovesyou
Created March 6, 2021 01:44
Show Gist options
  • Save piglovesyou/29c15f0d386f4343a5f956d1928044ae to your computer and use it in GitHub Desktop.
Save piglovesyou/29c15f0d386f4343a5f956d1928044ae to your computer and use it in GitHub Desktop.
Speed comparison of Node 15 file access with factors size and methods (sync, async and p-map concurrency)
import { readFileSync, writeFileSync, unlinkSync, promises } from "fs";
import pMap from "p-map";
const { readFile, writeFile, unlink } = promises;
const testCount = 10_000;
main().catch(err => (console.error(err), process.exit(1)));
async function main() {
const sizeBase = 1024;
for (let m = 1; m <= 2; m++) {
const size = Math.pow(sizeBase, m);
const content = Buffer.alloc(size, "0", "utf-8");
const unit =
m === 1 ? "K" :
m === 2 ? "M" :
m === 3 ? "G" : "never";
console.log(`===Test read,write,unlink file of 1${unit} content===`);
console.time(`fori + await 1${unit}`);
for (let i = 0; i < testCount; i++) {
const filename = `zero${i}.txt`;
await writeFile(filename, content);
await readFile(filename);
await unlink(filename);
}
console.timeEnd(`fori + await 1${unit}`);
const emptyArr = Array.from(Array(testCount)).map((_, i) => i);
const concurrencyBase = 2;
for (let mm = 1; mm <= 6; mm++) {
const concurrency = Math.pow(concurrencyBase, mm);
console.time(`pMap ${concurrency} concurrency 1${unit}`);
await pMap(emptyArr, async i => {
const filename = `zero${i}.txt`;
await writeFile(filename, content);
await readFile(filename);
await unlink(filename);
}, { concurrency });
console.timeEnd(`pMap ${concurrency} concurrency 1${unit}`);
}
console.time(`fori + sync 1${unit}`);
for (let i = 0; i < testCount; i++) {
const filename = `zero${i}.txt`;
writeFileSync(filename, content);
readFileSync(filename);
unlinkSync(filename);
}
console.timeEnd(`fori + sync 1${unit}`);
}
}
@piglovesyou
Copy link
Author

piglovesyou commented Mar 6, 2021

$ node -v
v15.10.0
$ uname
Darwin
$ sw_vers
ProductName:	Mac OS X
ProductVersion:	10.15.7
BuildVersion:	19H524
$ node my.mjs
===Test read,write and unlink file of 1K content===
fori + await 1K: 3.352s
pMap 2 concurrency 1K: 1.895s
pMap 4 concurrency 1K: 1.617s
pMap 8 concurrency 1K: 1.262s
pMap 16 concurrency 1K: 1.237s
pMap 32 concurrency 1K: 1.332s
pMap 64 concurrency 1K: 1.462s
fori + sync 1K: 2.400s
===Test read,write and unlink file of 1M content===
fori + await 1M: 28.126s
pMap 2 concurrency 1M: 19.575s
pMap 4 concurrency 1M: 16.944s
pMap 8 concurrency 1M: 15.733s
pMap 16 concurrency 1M: 15.852s
pMap 32 concurrency 1M: 15.642s
pMap 64 concurrency 1M: 18.173s
fori + sync 1M: 13.720s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment