Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Works a little bit like `Array.prototype.forEach()`, but uses `process.nextTick` to allow other processes to execute. NOTE. This isn't meant to be practical; if you use this in production code you're probably doing it wrong.
/**
* Allow other processes to execute while iterating over
* an array. Useful for large arrays, or long-running processing
*
* @param {Function} fn iterator fed each element of the array.
* @param {Function} next executed when done
*/
Array.prototype.nonBlockingForEach = function(fn, next) {
var arr = this;
var i = 0;
var len = arr.length;
function iter() {
if (i < len) {
fn(arr[i]);
i++;
process.nextTick(iter);
} else {
next();
}
}
iter();
};

dfkaye commented Sep 13, 2013

This is really nice ~ thanks for posting.

shovon commented Sep 14, 2013

One suggustion: you could name it as forEachAsync.

shovon commented Sep 14, 2013

I have recreated your function, and saved it as a gist.

My version is pretty much analogous to forEach, except async, where each element is iterated through asynchronously. The arguments list for the iterator corresponds with forEach, except that there is the added callback parameter.

I also ensured that the function handles errors as well. I follow the convention of how most Node.js developers implement asynchronous callbacks. That is, the first parameter of the callback expects an error object.

shovon commented Sep 14, 2013

But an asynchronous forEach is only a piece to the puzzle if you want to write truly asynchronous code. I might suggest using @caolan's async.js. Not only does it have an equivalent forEach function (simply called each in the library), it also has functions to help you run an array of asynchronous functions, and many more. I definitely recommend you checking it out.

Owner

neilk commented Sep 14, 2013

Oh, thanks. In my Node.js education (I just got deeply into it a few months ago) I skipped over @caolan's async.js and went straight to Promises, thinking they were new and shiny and modern. But it might be better to take a step back.

I was about to mention that you can use promises with existing Array builtins, but both that and the Async library are missing the point. The point is to free up CPU cycles during the iteration itself, under the assumption that the collection is large enough to cause the iteration itself to be slow.

Hello!
I've tryed this function, iterating over an array with a bilion of undefined elements (new Array(1000000000)),
inside one of my express js route handler, but when i try to visit that specific endpoint (for example /news, which handler start the long iteration), the all application is blocked, and other endpoints does not respond.
What am I missing?

G.

@gabrieledarrigo, the documentation for process.nextTick says

Note: the nextTick queue is completely drained on each pass of the event loop before additional I/O is processed. As a result, recursively setting nextTick callbacks will block any I/O from happening, just like a while(true); loop.

This is exactly what this function is doing so no wonder your express end points are not responding.

Use setImmediate instead of process.nextTick

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment