async graphs in js

async-in-js.md

Comparison of several ways of working with complex async graphs in js

  A     E
 /|\    |
B | C   F
 \|/
  D

This is dependency graph. Every letter is async task. Every line is dependency - from bottom to top. So D can be evaluated only after A, B and C.

Task complete only after all subtasks evaluated.

It's important to start tasks in parallel and as soon as possible.

Most popular approach is callbacks (not for this task but overall):

function (done) {
    var subtasksComplete = 0;
    var _d;
    AEval(function (err, a) {
        if (err) {
            return done(err);
        }
        var dDeps = {};

        function dDepsFullfiled() {
            DEval(a, dDeps.b, dDeps.c, function (err, d) {
                if (err) {
                    return done(err);
                }
                _d = d;
                subtasksComplete++;
                if (subtasksComplete === 2) {
                    done(null, d);
                }
            });
        }
        BEval(a, function (err, b) {
            if (err) {
                return done(err);
            }
            dDeps.b = b;
            if (dDeps.c) {
                dDepsFullfiled();
            }
        });
        CEval(a, function (err, c) {
            if (err) {
                return done(err);
            }
            dDeps.c = c;
            if (dDeps.b) {
                dDepsFullfiled();
            }
        });
    });
    EEval(function (err, e) {
        if (err) {
            return done(err);
        }
        FEval(e, function (err, f) {
            if (err) {
                return done(err);
            }
            subtasksComplete++;
            if (subtasksComplete === 2) {
                done(null, _d);
            }
        });
    });
}

Oh my callback hell! So shitty. If you can do it better - don't hesitate, post me please. But anyway unlikely it will be much better.

Sure you know about async.js

function (done) {
    async.auto({
        a: function (done) {
            AEval(done);
        },
        b: ['a', function (done, results) {
            BEval(results.a, done);
        }],
        c: ['a', function (done, results) {
            CEval(results.a, done);
        }],
        d: ['a', 'b', 'c', function (done, results) {
            DEval(results.a, results.b, results.c, done);
        }],
        e: function (done) {
            EEval(done);
        },
        f: ['e', function (done, results) {
            FEval(results.e, done);
        }]
    }, function (err, results) {
        done(err, results.d);
    });
}

Way better. Now it's actually possible to read the code and understand developer's intent.

You still have some problems related to callbacks, e.g. calling it twice.

But let's try Promise.

function () {
    var dP = AEval()
        .then(function (a) {
            var bP = BEval(a);
            var cP = CEval(a);
            return [a, bP, cP];
        })
        .spread(DEval)
    var fP = EEval()
        .then(FEval);
    return Promise.join(dP, fP, function (d) {
        return d;
    });
}

It isn't standart Promise but bluebird. async.js also isn't standart library, so in comparison there is no difference.

It's most common way to work with promises. I've seen this in some libs and projects.

It describes threads of async tasks nicely, so 'after A you can evaluate b and c' and so on.

But there is another one approach for Promises. Let's compare

function () {
    var aP = aEval();
    var bP = aP.then(bEval);
    var cP = aP.then(cEval);
    var dP = Promise.join(aP, bP, cP, DEval);
    var eP = EEval();
    var fP = eP.then(FEval);
    return Promise.join(dP, fP, function (d, f) {
        return d;
    });
}

Oh wow! So plain. Looks almost like syncronious code.

Sync... oh I heard about 'async looks like sync'.

co - your turn:

co(function* () {
    var a = yield aEval();
    var b = yield bEval(a);
    var c = yield cEval(a);
    var d = yield dEval(a, b, c);
    var e = yield eEval();
    var f = yield fEval(e);
    return d;
});

Just compare with first callback-based version! There is no better solution in the world, isn't it?

Ok, I hope I didn't catch you.

This so nice looking code doesn't runs in parallel (eEval runs only after dEval). It means that first version with callbacks in fact is better.

Here is the fixed version:

co(function* () {
    var dP = co(function* () {
        var a = yield aEval();
        var bP = bEval(a);
        var cP = cEval(a);
        var b = yield bP;
        var c = yield cP;
        return yield dEval(a, b, c);
    });
    var fP = co(function* () {
        var e = yield eEval();
        return yield fEval(e);
    });
    yield fP;
    return yield dP;
});

Not sure whether it possible to make it better. You know, help me if you can.

Hmm, but it doesn't look so nice as previous.

Some of you have questions:

"This stars* and yields - what does it mean?"

"Can I use it in browser?"

You have some troubles here.

In fact it's magic. I doubt that generators was created for this.

Summary

callback - hell

async - ok, but verbose, and it still callbacks

promise chains - nice for simple chains, but tricky for complex graphs

promise variables - best for complex graphs. Ok for simple

co - for simple async chain looks really nice. But also tricky for complex. And you have troubles with browsers

The Winner

I think Promises is the best way to describe complex async graphs.

Promises were created for that.

You can use it in node or browser without building step.

For simple chains you can use chains of then. In comparsion with co it will looks almost the same.

For complex async graphs it's better to create variable for each promise.

Please, ask questions, send corrections, suggest fixes.

Really want do discuss it.

Can be true?

import co from 'co';

let delay = value => new Promise((resolve, reject) => setTimeout(() => resolve(value), 1000));

// coroutine
co(function* () {

  let [a, e] = yield Promise.all([delay('A'), delay('E')]);
  console.log(a, e);
  let [b, c, f] = yield Promise.all([delay('B'), delay('C'), delay('F')]);
  console.log(b, c, f);

  return yield delay('D');

}).then(d => console.log(d));

// async/await
(async function () {

  let [a, e] = await Promise.all([delay('A'), delay('E')]);
  console.log(a, e);
  let [b, c, f] = await Promise.all([delay('B'), delay('C'), delay('F')]);
  console.log(b, c, f);
  let d = await delay('D');
  console.log(d);

})();

@evenfrost yes, it works. For me it looks like code with assumptions that isn't clear at all.

Version with promises very declarative: "what should be done before this can be done".

Try to make a change to deps graph: E can be done only after C.

function () {
    var aP = aEval();
    var bP = aP.then(bEval);
    var cP = aP.then(cEval);
    var dP = Promise.join(aP, bP, cP, DEval);
    var eP = cP.then(EEval);
    var fP = eP.then(FEval);
    return Promise.join(dP, fP, function (d, f) {
        return d;
    });
}

Just one line changed.

And compare with:

(async function () {

  let a = await delay('A');
  console.log(a);
  let [b, c] = await Promise.all([delay('B'), delay('C')]);
  console.log(b, c);
  let [d, e] = await Promise.all([delay('D'), delay('E')]);
  console.log(d, e);
  let f = await delay('F');

})();

Almost totally reworked.

Also I don't like introducing so many new syntax sugar for task that can be done with old simple es3.

async function task() {
    let a = await AEval();
    let b = await BEval(a);
    return b;
}

It's probably easier to work with this code.

Nice. But:

function task() {
    var aP = AEval();
    var bP = aP.then(BEval);
    return bP;
}

looks simpler. In both examples you should know about promises anyway.

I don't for refuse something because it should be learned. async/await is simple for that. It just isn't necessary, and even doesn't help at all

Well, some believe Promises are just 'intermediate' level on the road to ES7 async/await. And I think it's not about syntax sugar but about improvements in management of JS control flow. As you correctly mentioned, we can write all the stuff in old palmy ES3, though in this case (if no 3rd party libs are used) we'll fall into callback hell in any scenario. That's why promises were introduced in ES6, and that's why there is async/await spec in ES7. :)
And in my opinion _await_s look a bit cleaner than _then_ables, but everybody ought to do his own thing for sure.