Skip to content

Instantly share code, notes, and snippets.

Embed
What would you like to do?
List all files in a directory in Node.js recursively in a synchronous fashion
// List all files in a directory in Node.js recursively in a synchronous fashion
var walkSync = function(dir, filelist) {
var fs = fs || require('fs'),
files = fs.readdirSync(dir);
filelist = filelist || [];
files.forEach(function(file) {
if (fs.statSync(dir + file).isDirectory()) {
filelist = walkSync(dir + file + '/', filelist);
}
else {
filelist.push(file);
}
});
return filelist;
};
@winamp

This comment has been minimized.

Copy link

commented Sep 6, 2014

line 7 should look like the following:

if (fs.statSync(dir + '/' + file).isDirectory()) {
@VinGarcia

This comment has been minimized.

Copy link

commented Nov 26, 2014

@winamp the functions expect the dir name to end with '/', a better fix would be to check if dir is ending with '/', if not add it.

@VinGarcia

This comment has been minimized.

Copy link

commented Nov 26, 2014

@kethinov i made two changes on my fork:
I'm checking for '/' on the end of the dir name.
I'm returning the relative address of the file, instead of the file name.

pull request:
https://gist.github.com/VinGarcia/ba278b9460500dad1f50

@yesley

This comment has been minimized.

Copy link

commented Jul 23, 2015

I think that it will be better to use path.join(dir, file);
In this case function would handle both Unix/Mac and Windows evironments as they use different slashes in paths

@cneryjr

This comment has been minimized.

Copy link

commented Oct 23, 2015

Changed line 7 and 8 worked for me.

// List all files in a directory in Node.js recursively in a synchronous fashion
var walkSync = function(dir, filelist) {
  var fs = fs || require('fs'),
      files = fs.readdirSync(dir);
  filelist = filelist || [];
  files.forEach(function(file) {
    if (fs.statSync(dir + '/' + file).isDirectory()) {
      filelist = walkSync(dir + '/' + file, filelist);
    }
    else {
      filelist.push(file);
    }
  });
  return filelist;
};
@TifPun

This comment has been minimized.

Copy link

commented Oct 7, 2016

Made a small change about how file path is constructed. It's more robust to use path.join() then doing it manually:

// List all files in a directory in Node.js recursively in a synchronous fashion
var walkSync = function(dir, filelist) {
  var path = path || require('path');
  var fs = fs || require('fs'),
      files = fs.readdirSync(dir);
  filelist = filelist || [];
  files.forEach(function(file) {
    if (fs.statSync(path.join(dir, file)).isDirectory()) {
      filelist = walkSync(path.join(dir, file), filelist);
    }
    else {
      filelist.push(file);
    }
  });
  return filelist;
};
@ghostfreak3000

This comment has been minimized.

Copy link

commented Nov 14, 2016

Made a small change to return the absolute path for every file

     // List all files in a directory in Node.js recursively in a synchronous fashion
     var walkSync = function(dir, filelist) {
            var path = path || require('path');
            var fs = fs || require('fs'),
                files = fs.readdirSync(dir);
            filelist = filelist || [];
            files.forEach(function(file) {
                if (fs.statSync(path.join(dir, file)).isDirectory()) {
                    filelist = walkSync(path.join(dir, file), filelist);
                }
                else {
                    filelist.push(path.join(dir, file));
                }
            });
            return filelist;
        };
@biglovisa

This comment has been minimized.

Copy link

commented Dec 8, 2016

no functionality change but slightly readable and cleaner syntax

const walkSync = (dir, filelist = []) => {
  fs.readdirSync(dir).forEach(file => {

    filelist = fs.statSync(path.join(dir, file)).isDirectory()
      ? walkSync(path.join(dir, file), filelist)
      : filelist.concat(path.join(dir, file));

  });
return filelist;
}
@o1sound

This comment has been minimized.

Copy link

commented Dec 21, 2016

Modded this to structure the output as a directory tree.

// List all files [as directory tree] in Node.js recursively in a synchronous fashion
var walkSync = function(dir, filelist) {
    var fs = fs || require('fs'),
        files = fs.readdirSync(dir);
    filelist = filelist || [];
    files.forEach(function(file) {
        if (fs.statSync(dir + '/' + file).isDirectory()) {
            filelist.push(walkSync(dir + '/' + file, []));
        }
        else { filelist.push(file); }
    });
    return filelist;
};
@daviestar

This comment has been minimized.

Copy link

commented Jan 20, 2017

Directory tree map structure, where a file is represented by a string path, and a directory is represented by a object with key as path, and value is an array of children:

const allFilesSync = (dir, fileList = []) => {
  fs.readdirSync(dir).forEach(file => {
    const filePath = path.join(dir, file)

    fileList.push(
      fs.statSync(filePath).isDirectory()
        ? {[file]: allFilesSync(filePath)}
        : file
    )
  })
  return fileList
}
@ronnross

This comment has been minimized.

Copy link

commented Feb 18, 2017

If one had a preference to cut down on assignments the .forEach could be replaced with a map.

const walkSync = (dir, filelist = []) => fs.readdirSync(dir)
                                           .map(file => fs.statSync(path.join(dir, file)).isDirectory()
                                                        ? walkSync(path.join(dir, file), filelist)
                                                        : filelist.concat(path.join(dir, file))[0])
@hdn8

This comment has been minimized.

Copy link

commented Mar 24, 2017

Needed this for an electron app with a filedrop that could accept files or folders, altered to return a collection in the same format as a browser file chooser. Uses node-mime to get the mimetype.

mime = require('mime');

const walkSync = (dir, filelist = []) => {
  fs.readdirSync(dir).forEach(file => {
    filelist = fs.statSync(path.join(dir, file)).isDirectory()
      ? walkSync(path.join(dir, file), filelist)
      : filelist.concat({
          name: file,
          path: path.join(dir, file),
          size: fs.statSync(path.join(dir, file)).size,
          type: mime.lookup(file)
      });
  });
  return filelist;
}

@vidul-nikolaev-petrov

This comment has been minimized.

Copy link

commented Mar 25, 2017

function walkSync(dir) {
    return fs.lstatSync(dir).isDirectory()
        ? fs.readdirSync(dir).map(f => walkSync(path.join(dir, f)))
        : dir;
}
// more readable:
function walkSync(dir) {
    if (!fs.lstatSync(dir).isDirectory()) return dir;

    return fs.readdirSync(dir).map(f => walkSync(path.join(dir, f))); // `join("\n")`
}
@hdf

This comment has been minimized.

Copy link

commented Mar 25, 2017

Noticed a lack of flattening and windows support...

const flatten = arr => arr.reduce((acc, val) => 
      acc.concat(Array.isArray(val) ? flatten(val) : val), []);

Array.prototype.flatten = function() {return flatten(this)};

const walkSync = dir => fs.readdirSync(dir)
      .map(file => fs.statSync(path.join(dir, file)).isDirectory()
        ? walkSync(path.join(dir, file))
        : path.join(dir, file).replace(/\\/g, '/')).flatten();
@mgutz

This comment has been minimized.

Copy link

commented Mar 25, 2017

@hdf Strange that you didn't use arrow function for flatten.

nm, you wanted flatten context to be array

@mnpenner

This comment has been minimized.

Copy link

commented May 30, 2017

With flattening, no helper functions necessary:

const FileSystem = require('fs');
const Path = require('path');

function readDirR(dir) {
    return FileSystem.statSync(dir).isDirectory()
        ? Array.prototype.concat(...FileSystem.readdirSync(dir).map(f => readDirR(Path.join(dir, f))))
        : dir;
}
@dashawk

This comment has been minimized.

Copy link

commented Jun 23, 2017

how can we handle or trap the errors like if the directory does not exist?

@liddack

This comment has been minimized.

Copy link

commented Jun 23, 2017

You can also add file extension filters:

// ...
    if (fs.statSync(path.join(dir, file)).isDirectory()) {
      filelist = walkSync(path.join(dir, file), filelist);
    }
    else {
        // In case of video files...
        if (file.indexOf('.mp4') == file.length - 4
                || file.indexOf('.avi') == file.length - 4
                || file.indexOf('.m4v') == file.length - 4
                || file.indexOf('.mkv') == file.length - 4 {
            filelist.push(path.join(currentDir, file));
        }
    }
// ...
@reichert621

This comment has been minimized.

Copy link

commented Jul 21, 2017

Or just reduce! 🤓

const read = (dir) =>
  fs.readdirSync(dir)
    .reduce((files, file) =>
      fs.statSync(path.join(dir, file)).isDirectory() ?
        files.concat(read(path.join(dir, file))) :
        files.concat(path.join(dir, file)),
      []);

(or just use https://github.com/isaacs/node-glob)

@jackjwilliams

This comment has been minimized.

Copy link

commented Sep 9, 2017

This was probably the most interesting gist I've ever read, watching the syntax change over the years.

@A1rPun

This comment has been minimized.

Copy link

commented Sep 26, 2017

Amazing gists on this page. Had to leave a dirty one-liner.

const fs = require('fs');
const path = require('path');
const walkSync = (d) => fs.statSync(d).isDirectory() ? fs.readdirSync(d).map(f => walkSync(path.join(d, f))) : d;
@davidtorroija

This comment has been minimized.

Copy link

commented Nov 16, 2017

nice work thanks a lot! 🥇

@thereis

This comment has been minimized.

Copy link

commented Nov 24, 2017

I think I need to contribute with this snippet! I used this to to load all files inside folder and subfolders and exports them as a module.

"use strict"

const path = require('path')
const fs = require('fs')
const _ = require('lodash')

// recursive function to map files from directory
const walkSync = (d) => fs.statSync(d).isDirectory() ? fs.readdirSync(d).map(f => walkSync(path.join(d, f))) : d

_.forEach(walkSync(__dirname), (libraries) => {
    // avoid to include files inside the same folder
    if (_.isArray(libraries)) {
        _.forEach(_.flattenDeep(libraries), (lib) => {
            // check for eof
            if (lib.indexOf('.js') === -1) return
            // automatically exports the js file
            module.exports[lib] = require(lib)
        })
    }
})
@yiting007

This comment has been minimized.

Copy link

commented Jan 15, 2018

What if the directory tree is huge and may cause a stack overflow?

@bluedeck

This comment has been minimized.

Copy link

commented Mar 3, 2018

Guess then you'd let it overflow.

@x6herbius

This comment has been minimized.

Copy link

commented Mar 13, 2018

Given the above script already makes use of path, wouldn't it be better to use path.extname(lib) !== ".js", instead of lib.indexOf(".js") === -1? The latter would be fooled by not-a-lib.js.txt.

@FerreiraRaphael

This comment has been minimized.

Copy link

commented Mar 22, 2018

/**
 * Find all files inside a dir, recursively.
 * @function getAllFiles
 * @param  {string} dir Dir path string.
 * @return {string[]} Array with all file names that are inside the directory.
 */
const getAllFiles = dir =>
  fs.readdirSync(dir).reduce((files, file) => {
    const name = path.join(dir, file);
    const isDirectory = fs.statSync(name).isDirectory();
    return isDirectory ? [...files, ...getAllFiles(name)] : [...files, name];
  }, []);
@Skhmt

This comment has been minimized.

Copy link

commented May 21, 2018

Isn't there a race condition between fs.statSync(...).isDirectory() and the recursion?

Maybe instead:

const fs = require('fs');
const path = require('path');

function walkSync (dir, filelist = []) {
    fs.readdirSync(dir).forEach(file => {
        const dirFile = path.join(dir, file);
        try {
            filelist = walkSync(dirFile, filelist);
        }
        catch (err) {
            if (err.code === 'ENOTDIR' || err.code === 'EBUSY') filelist = [...filelist, dirFile];
            else throw err;
        }
    });
    return filelist;
}
@luciopaiva

This comment has been minimized.

Copy link

commented Aug 29, 2018

Forked it to make it a generator function so that space complexity goes from O(n) to O(1) (link to fork with full implementation here).

/**
 * List all files in a directory recursively in a synchronous fashion.
 *
 * @param {String} dir
 * @returns {IterableIterator<String>}
 */
function *walkSync(dir) {
    const files = fs.readdirSync(dir);

    for (const file of files) {
        const pathToFile = path.join(dir, file);
        const isDirectory = fs.statSync(pathToFile).isDirectory();
        if (isDirectory) {
            yield *walkSync(pathToFile);
        } else {
            yield pathToFile;
        }
    }
}
@quantumsheep

This comment has been minimized.

Copy link

commented Oct 15, 2018

Hi, here's a modern version:

const fs = require('fs').promises;
const path = require('path');

const walk = async (dir, filelist = []) => {
  const files = await fs.readdir(dir);

  for (file of files) {
    const filepath = path.join(dir, file);
    const stat = await fs.stat(filepath);

    if (stat.isDirectory()) {
      filelist = await walk(filepath, filelist);
    } else {
      filelist.push(file);
    }
  }

  return filelist;
}
@darekmeco

This comment has been minimized.

Copy link

commented Dec 1, 2018

Read as hierarchy tree

    const walkSync = (dir, filelist = []) => {
      const files = fs.readdirSync(dir);
      for (const file of files) {
        const dirFile = path.join(dir, file);
        const dirent = fs.statSync(dirFile);
        if (dirent.isDirectory()) {
          console.log('directory', path.join(dir, file));
          var odir = {
            file: dirFile,
            files: []
          }
          odir.files = walkSync(dirFile, dir.files);
          filelist.push(odir);
        } else {
          filelist.push({
            file: dirFile
          });
        }
      }
      return filelist;
    };
@chrisvoo

This comment has been minimized.

Copy link

commented Dec 27, 2018

Hi,
this is an excerpt from a project of mine to make a database of all the MP3 I have, using recursive-readdir. It takes around 40 mins for 18641 files. I was wondering how to shorten this time using multiple processes to scan different parts of the file system's portion.

        const promises = [];
        promises.push(readdir(thePath, [function ignoreFiles(file, stats) {
            if (stats.isDirectory()) {
                return false;
            }

            return !isMp3(file);
        }]));
@simlu

This comment has been minimized.

Copy link

commented Mar 17, 2019

Throwing another iterative (non recursive) one in here. It won't follow symlinks:

const fs = require('fs');
const path = require('path');

module.exports = (dir) => {
  const result = [];

  const files = [dir];
  do {
    const filepath = files.pop();
    const stat = fs.lstatSync(filepath);
    if (stat.isDirectory()) {
      fs
        .readdirSync(filepath)
        .forEach(f => files.push(path.join(filepath, f)));
    } else if (stat.isFile()) {
      result.push(path.relative(dir, filepath));
    }
  } while (files.length !== 0);

  return result;
};

Shameless self promotion. I now use smart-fs

@SamMaxwell

This comment has been minimized.

Copy link

commented May 6, 2019

an FP approach

const { join } = require('path');
const { readdirSync, statSync } = require('fs');
const {
  concat, difference, filter, map, reduce,
} = require('lodash/fp');

const getFilePaths = (path, encoding) => {
  const entries = readdirSync(path, { encoding });
  const paths = map(entry => join(path, entry), entries);
  const filePaths = filter(entryPath => statSync(entryPath).isFile(), paths);
  const dirPaths = difference(paths, filePaths);
  const dirFiles = reduce((prev, curr) => concat(prev, getFilePaths(curr)), [], dirPaths);
  return [...filePaths, ...dirFiles];
};

module.exports = getFilePaths;
@SirMorfield

This comment has been minimized.

Copy link

commented Jun 4, 2019

Async, EJS6, and showing full path.
As few packages as possible.
Tested in Node 10.16.0

const fs = require('fs').promises
const path = require('path')

async function walk(dir, fileList = []) {
  const files = await fs.readdir(dir)
  for (const file of files) {
    const stat = await fs.stat(path.join(dir, file))
    if (stat.isDirectory()) fileList = await walk(path.join(dir, file), fileList)
    else fileList.push(path.join(dir, file))
  }
  return fileList
}

walk('/home/').then((res) => {
  console.log(res)
})
@darioblanco

This comment has been minimized.

Copy link

commented Jun 6, 2019

@SamMaxwell approach without using lodash

const fs = require('fs');
const path = require('path');

/** Retrieve file paths from a given folder and its subfolders. */
const getFilePaths = (folderPath) => {
  const entryPaths = fs.readdirSync(folderPath).map(entry => path.join(folderPath, entry));
  const filePaths = entryPaths.filter(entryPath => fs.statSync(entryPath).isFile());
  const dirPaths = entryPaths.filter(entryPath => !filePaths.includes(entryPath));
  const dirFiles = dirPaths.reduce((prev, curr) => prev.concat(getFilePaths(curr)), []);
  return [...filePaths, ...dirFiles];
};

module.exports = getFilePaths;
@SirMorfield

This comment has been minimized.

Copy link

commented Jul 19, 2019

@jcollum-nike

@SirMorfield what version of Node is that running in?

Version 10.16.0 LTS.
I think this thread is not really ment for questions, please delete your question afterwards. I'll also remove this after you've read it.

@dmcgrandle

This comment has been minimized.

Copy link

commented Aug 6, 2019

rxjs 6.5, Typescript 3.5, node 10.6, modifying @quantumsheep's solution above, including both files and directories in a single pass, streaming results into an Observable of a custom type:

import { Stats, promises as fs } from 'fs';
import { Observable } from 'rxjs';

const dirsAndFiles = (dir: string): Observable<FileObject> =>
  new Observable(
    (observer): void => {
      // Create an async recursive function for walking the directory tree:
      const walk = async (dir: string): Promise<void> => {
        const files = await fs.readdir(join(__dirname, dir));
        for (const file of files) {
          const filepath = join(dir, file);
          const stat: Stats = await fs.stat(join(__dirname, filepath));
          if (stat.isFile() || stat.isDirectory()) {
            observer.next({
              isFile: stat.isFile(),
              filename: file,
              size: stat.size,
              path: dir
            });
          }
          if (stat.isDirectory()) {
            await walk(filepath);
          }
        }
      };
      // now call that async function and complete the Observable when it resolves
      walk(dir)
        .then((): void => observer.complete())
        .catch((err): void => observer.error(err));
    }
  );

// later in code:
dirsAndFiles(dir).subscribe(
  (file: FileObject) => doSomething(file),
  (err): void => processError(err)
);
@blaasvaer

This comment has been minimized.

Copy link

commented Aug 22, 2019

All fine, but how do you create an object than simulates the structure of the actual directory structure?

Like, from this:

Fig 1

LEVEL_1
	LEVEL_2
	|	LEVEL_3_1
	|	|	FILE_3_1_1
	|	|	FILE_3_1_2
	|	LEVEL_3_2
	|	|	FILE_3_2_1
	|	|	FILE_3_2_2
	|	|	LEVEL_4
	|	|	|	FILE_4_1
	|	|	|	FILE_4_2
	|	|	|	... this could go on forever ...
	|	FILE_2_1
	|	FILE_2_2
	FILE_1_1
	FILE_1_2

to this:

Fig 2

{
	LEVEL_2 : {
		LEVEL_3_1 : {
			FILE_3_1_1 : "FILE CONTENT",
			FILE_3_1_2 : "FILE CONTENT"
		},
		LEVEL_3_2 : {
			FILE_3_2_1 : "FILE CONTENT",
			FILE_3_2_2 : "FILE CONTENT"
			LEVEL_4 : {
				FILE_4_1 : "FILE CONTENT",
				FILE_4_2 : "FILE CONTENT"
			}
		},
		FILE_1_1 : "FILE CONTENT",
		FILE_2_1 : "FILE CONTENT"
	}
}
@geenloop

This comment has been minimized.

Copy link

commented Aug 26, 2019

good question, same here... anyone with ES5 old school simple solution?

@geenloop

This comment has been minimized.

Copy link

commented Aug 26, 2019

ouuu hell yea... I love google and I love open source comunity... works just like that:

https://www.npmjs.com/package/directory-tree

const dirTree = require("directory-tree");
const tree = dirTree("/Users/admin/Music");

console.log(tree)

thats all... the rest is up to Angular-tree-control xDDD good luck

*edit - solution #2 https://www.npmjs.com/package/dree NOT tested, just another option

@blaasvaer

This comment has been minimized.

Copy link

commented Aug 27, 2019

These return the structure alright in some 'weird' way, with all sorts of properties. My own solution to the problem with help from guys over at stackoverflow is more the kind I was looking for: https://stackoverflow.com/a/57626051/8082874

It return a straight object with folders as objects, filenames as keys – and in this case file contents as precompiled Handlebars templates ... like this:

{
	admin: {
		admin_subviews: {
			index: [Function]
		},
		header: {
			[Function: ret] isTop: true,
			_setup: [Function],
			_child: [Function]
		},
		index: {
			[Function: ret] isTop: true,
			_setup: [Function],
			_child: [Function]
		},
		layout: {
			[Function: ret] isTop: true,
			_setup: [Function],
			_child: [Function]
		}
	},
	index:{
		[Function: ret] isTop: true,
		_setup: [Function],
		_child: [Function]
	},
	layout:	{
		[Function: ret] isTop: true,
		_setup: [Function],
		_child: [Function]
	},
	login: {
		index: {
			[Function: ret] isTop: true,
			_setup: [Function],
			_child: [Function]
		},
		layout: {
			[Function: ret] isTop: true,
			_setup: [Function],
			_child: [Function]
		}
	}
}

The root level is the object itself and you can manipulate the file content in any way you want and inject it as the value.

@geenloop

This comment has been minimized.

Copy link

commented Aug 27, 2019

I have no idea what you mean but for me it just rendered JSON on server of all files/folders in tree -> send that tree to front-end-> and https://github.com/wix/angular-tree-control just directly render all of that at a glance 👍 but you look like that you need more than that...

@blaasvaer

This comment has been minimized.

Copy link

commented Aug 27, 2019

Probably yes. And whenever something contains 'angular' I'm usually out anyway, so ; )

@geenloop

This comment has been minimized.

Copy link

commented Aug 27, 2019

yea xD I'm locked in the 2015 year development and ES5 but I'm so grateful for that but depends on everybody's needs ;) good luck anyway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.