Skip to content

Instantly share code, notes, and snippets.

@jkrems
Last active February 24, 2020 19:09
  • Star 62 You must be signed in to star a gist
  • Fork 0 You must be signed in to fork a gist
Star You must be signed in to star a gist
Save jkrems/04a2b34fb9893e4c2b5c to your computer and use it in GitHub Desktop.
Generators Are Like Arrays

In all the discussions about ES6 one thing is bugging me. I'm picking one random comment here from this io.js issue but it's something that comes up over and over again:

There's sentiment from one group that Node should have full support for Promises. While at the same time another group wants generator syntax support (e.g. var f = yield fs.stat(...)).

People keep putting generators, callbacks, co, thunks, control flow libraries, and promises into one bucket. If you read that list and you think "well, they are all kind of doing the same thing", then this is to you.

There are three distinct categories that they fall into:

  1. Models/Abstractions of async behavior
  2. Control flow management
  3. Data structures

Generators are category 3. Genators are like arrays. Don't believe me? Here's some code:

function *doStuff() {
  yield fs.readFile.bind(null, 'hello.txt');
  yield fs.readFile.bind(null, 'world.txt');
  yield fs.readFile.bind(null, 'and-such.txt');
}

What does it do? Not much actually. It just creates a list of functions that take a callback. We can even iterate over it:

for (task of doStuff()) {
  // task is a function that takes a good ol' callback
}

Written down using boring, ES5 code:

function doStuff() {
  return [
    fs.readFile.bind(null, 'hello.txt'),
    fs.readFile.bind(null, 'world.txt'),
    fs.readFile.bind(null, 'and-such.txt')
  ];
}

Ready to get your mind blown? We can use the exact same for-loop snippet to iterate over this.

Now, of course generators aren't just a slightly more verbose array syntax. They allow you to dynamically alter the content of the array based on stuff being passed in or to return lazy (read: infinite) sequences. All this can be done in ES5 already (regenerator is proof of that), but generators do offer a nicer syntax.

But they aren't async. And they don't manage control flow. Would you say that an array is "control flow management" because you can pass it into async.series?

async.series(doStuff());

Then why would you call generators "control flow management" because you can pass one into co?

co(doStuff());

Sure, generators are a more powerful "data structure" than arrays. But they are still closer to arrays than they are to promises, caolan/async, or callbacks.

If you want to do something async, you need category 1 (an abstraction of async behavior). To make it nicer, category 2 (control flow management) can be helpful. And more often than not category 2 will require you to use known data structures from category 3. You can pick your poison for category 1 and 2 freely. But you won't be able to replace a promise with a fancier array.

  1. Models/Abstractions for async behavior: thunks+callbacks, promises
  2. Control flow management: co, async, spawn/ES7 async functions, Promise.all
  3. Data structures: arrays, generators, objects/maps

P.S.: I hope this post can usher in an era of JS developers using all sorts of different, slightly weird analogies to explain an often misunderstood language feature. Then we finally got our own little "Monads are like X".

P.P.S: The right choice is obviously Promises + async functions.

@sgoguen
Copy link

sgoguen commented Dec 5, 2014

I think the word you're looking for is that generators are enumerable, in that generators support an enumerating interface like arrays. Apart from that, they diverge in many ways.

@jmar777
Copy link

jmar777 commented Dec 5, 2014

I went into more detail in this in a blog post, but more succinctly, the big deal with generators and asynchrony can be illustrated in a short example like this:

function* helloWorldGenerator() {
    yield 'hello';
    yield 'world';
}

var hw = helloWorldGenerator();
console.log(hw.next()); // prints { value: 'hello', done: false }
setTimeout(function() {
    console.log(hw.next()); // prints { value: 'world', done: false }
}, 1000);

As stated in the post I referenced:

we have a full one-thousand beautiful milliseconds between yield 'hello' and yield 'world', and yet those lines of code are written in a very synchronous-looking syntax. This is a big deal: generators finally provide us with a pseudo-synchronous syntax that doesn't break run-to-completion semantics, doesn't require transpiling, and doesn't require callbacks.

Before generators, it didn't matter if you were using callbacks, continuations, events, Promises, etc., you always had to supply a function somewhere to get control back after an asynchronous operation. Whether you view it as a hacky solution or not, generator-based control-flow libraries like suspend and co really do change things.

Especially when coupled with Promises, they enable valid ES6 code to very closely resemble what (speculative) ES7 code will look like:

// ES6
suspend(function*(id) {
    var user = yield User.findById(id).fetch();
});

// ES7
async function(id) {
    var user = await User.findById(id).fetch();
}

@jkrems
Copy link
Author

jkrems commented Dec 5, 2014

@jmar777 Well, that example is not really convincing...

var helloWorldGenerator = [ 'hello', 'world' ];
var hw = helloWorldGenerator[Symbol.iterator](); // guessing the details here
console.log(hw.next()); // prints { value: 'hello', done: false }
setTimeout(function() {
    console.log(hw.next()); // prints { value: 'world', done: false }
}, 1000);

Btw. - see my P.P.S.: I'm a big fan of async functions. I'd rather use Bluebird (or 6to5) than co for them because they don't have the baggage that co has, which started of with that silly thunk thing. Nowhere in my article do I say that you can't use generators in combination with promises to do awesome things. I only said that you can't replace the promise-part with generators.

@inikulin
Copy link

inikulin commented Dec 6, 2014

Hi there!

There are three distinct categories that they fall into:

  1. Models/Abstractions of async behavior
  2. Control flow management
  3. Data structures

"Models/Abstractions of async behavior" are made of "Control flow management" and "Data structures". Therefore, there are two distinct categories.

Generators are category 3. Genators are like arrays.

No, they are not. Generators are routines ("semicoroutines" to be precise) and they don't have underlying data. They just provide the strategy to iterate over the quantity of any nature. So, generators fall in the category 2 in your classification.

Summarizing, I think your argument is built on top of false assumptions. Using generators for the async control flow is completely OK, since they are weak case of the coroutines which are established as the good control flow primitive to deal with async behavior.

@domenic
Copy link

domenic commented Dec 6, 2014

Generators are like arrays; generator functions are functions which have syntactic constructs that help you build generators. (Normal functions can only "build" a single value or exception; generator functions can build a generator, i.e. an iterable.)

@inikulin
Copy link

inikulin commented Dec 6, 2014

@domenic
Spec doesn't agree with you:

First-class coroutines, represented as objects encapsulating suspended execution contexts (i.e., function activations). Prior art: Python, Icon, Lua, Scheme, Smalltalk.

Yes, generator function produces iterator, but it's definitely not array, the closest analogy is the singly linked list. No random access, no access to the quantity size while iterating. Moreover, iterable quantity can be lazy evaluated, this makes it distinct from the linked list. So, I think that generators are arrays is the very misleading statement.

@jkrems
Copy link
Author

jkrems commented Dec 6, 2014

@inikulin The post doesn't say "generators are arrays". It compares them with arrays, in the context of async control flow management. It even explicitly says "of course generators aren't just a slightly more verbose array syntax".

@jmar777
Copy link

jmar777 commented Dec 6, 2014

@jkrems I'm not sure I understand your response; it doesn't address my comments regarding the significance of having a pseudo-synchronous syntax for async operations (i.e., getting control back from an async operation w/out a function being passed around somewhere).

@jkrems
Copy link
Author

jkrems commented Dec 7, 2014

@jmar777 Not sure I understand - you are passing hw around which contains references to the next actions to take. Or are you talking about Promises?

EDIT: If you just wanted to say that you like async functions (Promises + spawn + generators) then I'm not sure how that is relevant as a "rebuttal" of this gist. It's the exact thing I call out as my favorite in the P.P.S..

@jmar777
Copy link

jmar777 commented Dec 8, 2014

@jkrems Ahh, I wasn't really intending to rebut anything in your post; just attempting to clarify / add some context around why generators (combined with a runner like co or suspend) were immediately latched onto for control-flow management.

The example with hw has nothing to do with control flow on its own, it's just an example of how generator functions behave (and how there's some syntactical significance to the fact that a yield expression can span multiple turns on the event loop). Definitely nothing about Promises in that example.

@hollowdoor
Copy link

@jkrems Technically generators are not like arrays because yield is a type of return statement.

Semantically generators are like arrays because you can loop them.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment