An open question (rant) about node.js

nodejs-question.md

Most developers would agree that, all other things being equal, a synchronous program is easier to work with than an asynchronous one. The logic for this is pretty clear: one flow of execution is easier for the human mind to simulate than n concurrent flows.

After doing two small projects in node.js (one of which is here -- ready for the blinding flurry of criticism), there's one question that I can't shake: if asynchronicity is an optimization (that is, a complexity introduced for the sake of performance), why would people, a priori, turn to a framework that imposes it for everything? If asynchronous code is harder to reason about, why would we elect to live in a world where it is the default?

It could be argued pretty well that the browser is a domain that inherently lends itself to an async model, but I'd be very curious to hear a defense of "async-first" thinking for problems that are typically solved on the server-side. When working with node, I've noticed many regions of code where

1. synchronicity wouldn't introduce a performance bottleneck, and
2. what would otherwise be an easy problem is made very difficult by the fact that everything must be phrased for the event loop.

For an example of this, try writing a function call that requires information from two separate HTTP API responses; I basically need to draw a diagram of what happens with async.waterfall for a task that, given synchronicity, would've been solved with a trivial three-liner.

Easy things should be easy. Optimizations should be closeted until they're needed. Maybe I'm missing something here, some mechanism in node that allows opt-in synchronicity... dear node.js, is there such a thing? If not, why do you want to make many things harder than they need to be?

  if asynchronicity is an optimization (that is, a complexity introduced for the sake
  of performance), why would people, a priori, turn to a framework that imposes it for
  everything? If asynchronous code is harder to reason about, why would we elect to
  live in a world where it is the default?

Node.js imposes asynchronous operation for nearly everything because nearly everything meaningful requires at least a few IO operations, and anywhere you have IO you can be doing other things while you wait for a response. Anywhere you write to disk, make a web request, send out a Redis command, execute a database query rather than passively waiting for a response to come back you could be doing something else instead.

In the case of a web server, you can start answering another request while waiting for a database query for your previous request to finish. That is the power of Node.js, and what allows it handle thousands of concurrent connections to get maximum requests per second served per machine.

 I basically need to draw a diagram of what happens

Give it a few months and you'll be naturally coding in async style with no need for diagrams. Perhaps use async.auto for the time being. It's really easy to understand because you are basically just defining data dependencies for each step and letting async.auto handle the flow control to ensure that each step has the data it needs when it gets executed.

Look at this https://github.com/visionmedia/co
It allow you to write non-blocking code in synchronicity style

Erlang provides a far better set of primitives as a language for concurrent programming, without the headaches of shared state (except when you really need it). But it has a fraction of the packages that Node does, and doesn't have the same composability. I dream of a place somewhere between the two languages.

Asynchronous scripting takes some time to get used to, but when you get used to it, it's not any harder then reading any other language that has functions / subroutines / jumping to labels.

function cakeReady(cake) {
·· eat(cake);
}

bakeCake(cakeReady);

But if you really want to read from line to line and not jump between functions, you can in-line the function:

bakeCake(function cakeReady(cake) {
·· eat(cake);
});

And some people like to add the word "then" to make it easier to follow (aka "promises")

bakeCake().then(cakeReady);

You can also use the OO-pattern:

var cake = new Cake();
cake.ready = cakeReady;

And some people prefer:

cake.on("ready", cakeReady);

My feeling is that async programming with callbacks has a learning curve, but once you get it, then you get it. You won't generally have trouble with it again.

For me it took about a week of working on a series of nested for loops making asynchronous calls via github's api before it really sunk in. Yeah, it was frustrating working on that problem, but when it all came together, it made a lot of sense and that process probably made me a better programmer. Plus, that same series of calls done synchronously would have taken an unacceptably long amount of time (I was making between 10-300 calls). So if I was using ruby, I would have had to look into some kind of non-blocking technique in ruby (which I'm not familiar with).

Regarding promises, I think they're useful but overrated. They don't add functionality, they just make things prettier and easier for someone else to grasp what's going on in your code (promises are basically syntactic sugar for callbacks). Obviously those are both important things, but I think it's a mistake to use promises as an excuse to avoid learning asynchronous code flow. If you want to use node a lot, you're going to have to face asynchronous thinking eventually (or at least I did in my first week with node).

For example, I'd argue that Q.all([a,b]).then(c); IS much different than a(); b(); c();.