Skip to content

Instantly share code, notes, and snippets.

@timoxley
Last active August 6, 2023 17:13
Show Gist options
  • Save timoxley/0cb5053dec107499c8aabad8dfd651ea to your computer and use it in GitHub Desktop.
Save timoxley/0cb5053dec107499c8aabad8dfd651ea to your computer and use it in GitHub Desktop.
async/await recursive fs readdir
import { join } from 'path'
import { readdir, stat } from 'fs-promise'
async function rreaddir (dir, allFiles = []) {
const files = (await readdir(dir)).map(f => join(dir, f))
allFiles.push(...files)
await Promise.all(files.map(async f => (
(await stat(f)).isDirectory() && rreaddir(f, allFiles)
)))
return allFiles
}
// straight synchronous version for perf comparison
import { join } from 'path'
import fs from 'fs'
function rreaddirSync (dir, allFiles = []) {
const files = fs.readdirSync(dir).map(f => join(dir, f))
allFiles.push(...files)
files.forEach(f => {
fs.statSync(f).isDirectory() && rreaddirSync(f, allFiles)
})
return allFiles
}
import assert from 'assert'
console.time('parallel')
rreaddir(__dirname).then(pFiles => {
console.timeEnd('parallel')
console.time('sync')
const sFiles = rreaddirSync(__dirname)
console.timeEnd('sync')
assert.deepEqual(pFiles.sort(alpha), sFiles.sort(alpha))
})
function alpha (a, b) {
return a.localeCompare(b)
}
parallel: 313.899ms
sync: 80.259ms
@puckey
Copy link

puckey commented Feb 22, 2020

❤️

@puckey
Copy link

puckey commented Feb 22, 2020

Seemingly not much difference in performance, which is good:

queue: 935.704ms
sync: 1754.959ms
parallel: 945.062ms

Interesting to see that the parallel version is now much faster than the original sync version. Tested using Node v12.13.0 with slightly adjusted versions of parallel and sync (because my version also leaves out directories in the resulting file list).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment