Generators Are Like Arrays

generators.md

In all the discussions about ES6 one thing is bugging me. I'm picking one random comment here from this io.js issue but it's something that comes up over and over again:

There's sentiment from one group that Node should have full support for Promises. While at the same time another group wants generator syntax support (e.g. var f = yield fs.stat(...)).

People keep putting generators, callbacks, co, thunks, control flow libraries, and promises into one bucket. If you read that list and you think "well, they are all kind of doing the same thing", then this is to you.

There are three distinct categories that they fall into:

1. Models/Abstractions of async behavior
2. Control flow management
3. Data structures

Generators are category 3. Genators are like arrays. Don't believe me? Here's some code:

function *doStuff() {
  yield fs.readFile.bind(null, 'hello.txt');
  yield fs.readFile.bind(null, 'world.txt');
  yield fs.readFile.bind(null, 'and-such.txt');
}

What does it do? Not much actually. It just creates a list of functions that take a callback. We can even iterate over it:

for (task of doStuff()) {
  // task is a function that takes a good ol' callback
}

Written down using boring, ES5 code:

function doStuff() {
  return [
    fs.readFile.bind(null, 'hello.txt'),
    fs.readFile.bind(null, 'world.txt'),
    fs.readFile.bind(null, 'and-such.txt')
  ];
}

Ready to get your mind blown? We can use the exact same for-loop snippet to iterate over this.

Now, of course generators aren't just a slightly more verbose array syntax. They allow you to dynamically alter the content of the array based on stuff being passed in or to return lazy (read: infinite) sequences. All this can be done in ES5 already (regenerator is proof of that), but generators do offer a nicer syntax.

But they aren't async. And they don't manage control flow. Would you say that an array is "control flow management" because you can pass it into async.series?

async.series(doStuff());

Then why would you call generators "control flow management" because you can pass one into co?

co(doStuff());

Sure, generators are a more powerful "data structure" than arrays. But they are still closer to arrays than they are to promises, caolan/async, or callbacks.

If you want to do something async, you need category 1 (an abstraction of async behavior). To make it nicer, category 2 (control flow management) can be helpful. And more often than not category 2 will require you to use known data structures from category 3. You can pick your poison for category 1 and 2 freely. But you won't be able to replace a promise with a fancier array.

1. Models/Abstractions for async behavior: thunks+callbacks, promises
2. Control flow management: co, async, spawn/ES7 async functions, Promise.all
3. Data structures: arrays, generators, objects/maps

P.S.: I hope this post can usher in an era of JS developers using all sorts of different, slightly weird analogies to explain an often misunderstood language feature. Then we finally got our own little "Monads are like X".

P.P.S: The right choice is obviously Promises + async functions.

@domenic
Spec doesn't agree with you:

First-class coroutines, represented as objects encapsulating suspended execution contexts (i.e., function activations). Prior art: Python, Icon, Lua, Scheme, Smalltalk.

Yes, generator function produces iterator, but it's definitely not array, the closest analogy is the singly linked list. No random access, no access to the quantity size while iterating. Moreover, iterable quantity can be lazy evaluated, this makes it distinct from the linked list. So, I think that generators are arrays is the very misleading statement.

@inikulin The post doesn't say "generators are arrays". It compares them with arrays, in the context of async control flow management. It even explicitly says "of course generators aren't just a slightly more verbose array syntax".

@jkrems I'm not sure I understand your response; it doesn't address my comments regarding the significance of having a pseudo-synchronous syntax for async operations (i.e., getting control back from an async operation w/out a function being passed around somewhere).

@jmar777 Not sure I understand - you are passing hw around which contains references to the next actions to take. Or are you talking about Promises?

EDIT: If you just wanted to say that you like async functions (Promises + spawn + generators) then I'm not sure how that is relevant as a "rebuttal" of this gist. It's the exact thing I call out as my favorite in the P.P.S..

@jkrems Ahh, I wasn't really intending to rebut anything in your post; just attempting to clarify / add some context around why generators (combined with a runner like co or suspend) were immediately latched onto for control-flow management.

The example with hw has nothing to do with control flow on its own, it's just an example of how generator functions behave (and how there's some syntactical significance to the fact that a yield expression can span multiple turns on the event loop). Definitely nothing about Promises in that example.

@jkrems Technically generators are not like arrays because yield is a type of return statement.

Semantically generators are like arrays because you can loop them.