Skip to content

Instantly share code, notes, and snippets.

What would you like to do?
An open question (rant) about node.js

Most developers would agree that, all other things being equal, a synchronous program is easier to work with than an asynchronous one. The logic for this is pretty clear: one flow of execution is easier for the human mind to simulate than n concurrent flows.

After doing two small projects in node.js (one of which is here -- ready for the blinding flurry of criticism), there's one question that I can't shake: if asynchronicity is an optimization (that is, a complexity introduced for the sake of performance), why would people, a priori, turn to a framework that imposes it for everything? If asynchronous code is harder to reason about, why would we elect to live in a world where it is the default?

It could be argued pretty well that the browser is a domain that inherently lends itself to an async model, but I'd be very curious to hear a defense of "async-first" thinking for problems that are typically solved on the server-side. When working with node, I've noticed many regions of code where

  1. synchronicity wouldn't introduce a performance bottleneck, and
  2. what would otherwise be an easy problem is made very difficult by the fact that everything must be phrased for the event loop.

For an example of this, try writing a function call that requires information from two separate HTTP API responses; I basically need to draw a diagram of what happens with async.waterfall for a task that, given synchronicity, would've been solved with a trivial three-liner.

Easy things should be easy. Optimizations should be closeted until they're needed. Maybe I'm missing something here, some mechanism in node that allows opt-in synchronicity... dear node.js, is there such a thing? If not, why do you want to make many things harder than they need to be?

Copy link

vjpr commented Mar 14, 2014

I've felt your pain. Callbacks for control flow are hard to refactor and reason about. I find async and promises frustrating.

I use IcedCoffeeScript for all my Node.js code front and back. Makes writing async code a pleasure. Feels the same as coding sync.

et = require 'errTo'
{get} = require 'request'
fn = (done) ->
  await get '', et done, defer resp, body
  await get '', et done, defer resp, body
  do done
await fn defer err
throw err if err

Copy link

agentultra commented Mar 14, 2014

@meryn to be fair, Python can do everything Node does. See the various asynchronous-IO event loops: twisted, tornado, asyncio/tulip. I believe Ruby has equivalents as well. It's not a feature or methodology unique to Node.js

Copy link

nfour commented Mar 14, 2014

+1 to Promises + Generators. It just feels right. While it's currently a bit of a hassle to begin using them, it is ultimately what JS will trend into. The more people building applications in this way the sooner that future will come.

Copy link

scriby commented Mar 14, 2014

It sounds like you want Fibers (or upcoming generators). I'll plug my own flow control library built on fibers here:

Copy link

aashishkoirala commented Mar 14, 2014

Seems like Node could benefit from the equivalent of the async/await keywords in C#.

In a nutshell, with a few keywords, you can tell the compiler to convert code of the type:




Copy link

tqwhite commented Mar 14, 2014

Async allows much better performance/elegance in a substantial category of things. It is better to have one paradigm than two. It is my intention to suffer through the pain of turning myself into a 100% async thinker, treating the desire to return to linear programming as laziness, the confusion as inexperience.

I've been doing this a long, long time. I found myself suffering similar pain when the world turned object oriented after a career of procedural code. It was worth it. This will be, too. It also might hurt just a little bit more.

Copy link

mjhea0 commented Mar 14, 2014

Control flow is an interesting problem with Node and asynchronous frameworks in general.

We're going to be addressing this issue at Monday's meetup -

  1. We started with this app - (which pulls tweets and performs sentiment; I used a timeout to control the flow, which is obviously not the right way to go.
  2. We are going to be looking at -
  3. IceCoffee looks interesting. Maybe we'll look at that in the future.

Anyway - I think we all feel your pain. I for one don't even want to deal with callbacks, so promises is the most "promising" for me.

Great discussion!

Copy link

ulisesrmzroche commented Mar 14, 2014

@agentultra, if you don't find the idea of handling async events in Python/Ruby something really unpleasant, then you haven't done enough of it.

Copy link

christophercliff commented Mar 14, 2014

If your point is "async patterns are sometimes useful but async.waterfall is a bummer", then I agree but I think you miss the point.

Here's my take on the node.js story: A handful of Web technologies were combined to create a platform that was pretty good at solving Web programming problems. Turns out the platform appealed to TONS of people, and was useful in a lot of other areas as well. Then those tons of people created tons of value on the platform and gave it away (via Github, npm, etc.).

I turn to node.js because I mostly do Web stuff. The event loop is generally a good fit for my problems and I want to take advantage of all that community value. The async patterns and the JavaScript idiosyncrasies are at worst a minor annoyance. At best, they're exactly what I need.

Copy link

clipperhouse commented Mar 14, 2014

As with most languages, Node is a set of opinions (or defaults). One of those opinions is, it’s hard to block. Blocking is done very deliberately. One takes on the cognitive overhead of event-loop thinking because one believes it pays off.

One would choose Node because this is an attractive notion. If it’s not an attractive notion, then Node is a bad choice.

Copy link

inanna-malick commented Mar 14, 2014

For an example of this, try writing a function call that requires information from two separate HTTP API responses; I basically need to draw a diagram of what happens with async.waterfall for a task that, given synchronicity, would've been solved with a trivial three-liner.

It's a lot easier to handle asynchronicity in strongly-typed languages with monadic for comprehensions. In Scala, for example:

val res: Future[Foobar] = for {
  a <- makeHttpRequestA()
  b <- makeHttpRequestBFromA(a)
} yield new Foobar(a, b)

You can then register callback functions which operate on either a Success[Foobar] or a Failure[Throwable]. This is much simpler then using callbacks for everything

Synchronous code isn't simpler, it just hides complexity like caltrops in tall grass. You'll realize this when you try to scale up.

Copy link

tarcieri commented Mar 14, 2014

These sort of concerns are exactly why I wrote Celluloid the way I did:

Celluloid uses coroutines to abstract over asynchronous operations so you don't end up doing a CSP transformation by hand. The Celluloid::IO library provides duck types of Ruby's native IO classes, so if you can dependency inject these into existing libraries, you can use them in conjunction with an asynchronous event loop. There's no need to rewrite everything from the ground up in CSP-by-hand format.

Furthermore, an async event loop benefits one particular type of server: one that handles large numbers of mostly idle connections. Servers that handle a small number of highly active connections are better served by threads and blocking I/O managed by the kernel. Fortunately, Celluloid::IO's handles can be used transparently between both actors with an async I/O event loop and normal Ruby threads, so you can mix-and-match blocking I/O with async I/O.

Copy link

Warbo commented Mar 14, 2014

Asynchronous code isn't necessarily harder to think about, since we usually don't care about the order/interleaving of things; all we care about is that our code will eventually be run, with whatever data it needs (eg. a HTTP response). Sync/async is an implementation detail.

What can be difficult is trying to write down asynchronous code in a language like Javascript, since most of its syntax is hard-coded for doing synchronous stuff. This a) forces us to distinguish our async code using various wrappers and b) forces us to jump back into the synchronous world to actually get anything done (eg. handling results), since all of the built-in stuff we might want (arithmetic, loops, branches, array-lookup, etc.) is synchronous.

This makes async look more difficult than synchronous, but it's just an artifact of using a synchronous language and being forced to oscillate back-and-forth between the two.

Copy link

bwindels commented Mar 14, 2014

Yes, going from a synchronous style to an asynchronous style can have a steep learning curve and has some extra complexity, especially related to exception handling. But I also feel that it makes reasoning about concurrency easier since you typically only use one thread, and multiple threads (webworkers or forked processes) never share state.

Copy link

jakobmattsson commented Mar 14, 2014

IMHO, the main problem with promises is that people only use them half-way. Properly leveraged, all the async goes away and it looks exactly like sync code written in ruby or whatever, but automatically optimized to run async behind the scenes.

People keep missing that point and writing "then" all over their code, turning "callback-hell" into "then-hell".

I gave a presentation on this topic recently: Jump right to slide 40 if you want to see what promises should look like. Compare with the 4 pages preceding it.

The lib referred to can be found here:

Copy link

mjhea0 commented Mar 14, 2014

Copy link

felixhammerl commented Mar 14, 2014

var after = _.after(3, function() {
    // your calls are done


going from sync to async code is like saying: "ok, i want those three things (see above) to be done. don't really care how and when, just tell me when they're done."

doing this synchronously enforces an order where there is no order...

Copy link

julik commented Mar 14, 2014

Because even the promised ES6 features for async programming are vastly inadequate.

Copy link

joeabrams026 commented Mar 14, 2014

A few thoughts:

  1. After about 6 months working with node, I actually like async programming a lot. But, before those 6 months I was constantly looking for ways to do things more synchronously. So maybe it's just an adjustment period. I find that a lot of programming problems are actually async in nature (e.g. webapps) and map well to node. I therefor somewhat disagree with your statement that synchronous is easier to work with than asynchronous.
  2. I used to work on a major web application written in C#. Most of our worst 'production' bugs boiled down to blocking on I/O. If we had used node, most of those bugs would not have existed because our developers would not have physically been able to write blocking code.
  3. If I'm going to write something very synchronous I'll prefer to use a language with more syntactic sugar like python.

Copy link

jwarkentin commented Mar 14, 2014

There's something everyone seems to be missing. You're thinking that the asynchronous nature of JavaScript was meant as a performance optimization. Let's go back to JavaScript's origins. It was originally a scripting language for the browser. If it were to be synchronous and/or allow blocking things, such as sleep() or whatever else, it would have created an unusable web experience. Web pages would be constantly locking up. It's only recently that it's moved to the server where the asynchronicity on I/O isn't so mandatory.

I personally still think it's a good thing, but that could definitely be debated on the server side. Unfortunately, if you want to have the advantage of writing one code base that can run on the browser or server, then it must work the same in both places.

Copy link

timjansen commented Mar 14, 2014

In many cases synchronicity is not the alternative to asynchronous programming - multi-threading is. And if asynchronous programming is 5x harder than synchronous programming, then multi-threaded programming is 100x harder to get right.

Most non-trivial applications need to respond to several things at the same time. Like a server that needs to handle two simultanous clients, or a GUI application that needs to manage a user interface while at the same time downloading a file. For this kind of application synchronous programming is just not an option.

Now you could use multi-threading, which looks like synchronously code at first glance. But getting the communication between parallel threads right is really hard. The problems start with relatively well understood things like race-conditions, and end with arcane stuff like synchronizing memory between threads/cores. All those problems have in common that they cause bugs that can be subtle, but very hard to reproduce and debug. Compared to that, asynchronous programming is trivial.

(and yes, there are other alternatives: message passing, transactional memory... they are more powerful, but also more complex and more difficult to understand than Node's asynchronous model)

Copy link

focusaurus commented Mar 14, 2014

I don't think node.js will ever be a slam-dunk "first choice because it is easy and correct and consistent" system. In my mind it is a very pragmatic choice based on certain realities at the moment which only become compelling in combination and until something better arrives. Some of these include: same language as browser, v8 is very fast, the evented model has really proven itself superior in terms of concurrent connection efficiency for network servers (at least IMHO), the way npm works is vastly superior to all prior systems (pip, ruby gems, bundler, etc), the node community is building modern libraries in a way that (IMHO) is overall better than rubygems or pypi. Not that rubygems wasn't successful/effective, it's just that npm and the packages therein tend to be even better in being smaller, more flexible, easier to swap in and out, etc. Yes, asynchronous code is a big learning curve. No you should not go start converting your shell scripts to node.js just because. However, if you are writing a network server, node.js makes sense in many cases. But yeah if you need a CRUD app that you and 3 of your friends can use to share cookie recipes, the actual control flow coding will be easier in a synchronous launguage like Ruby or Python.

Copy link

fredguest commented Mar 14, 2014

Copy link

refractalize commented Mar 14, 2014

Take a look at pogoscript, it rewrites synchronous into asynchronous calls:

fs = require 'fs'
mojo = file 'mojo.txt' 'utf-8'!
console.log (mojo)

See pogoscript concurrency patterns for more examples.

Copy link

cheery commented Mar 14, 2014

I heard from somebody that node.js is inefficiently implemented. So whatever performance gains they get with async would be lost into sloppy implementation and lack of understanding about simple functions.

I keep using it, because sometimes it's just convenient to run javascript from the terminal. I don't see it as much as a framework. It's just yet another javascript target.

Copy link

nathanpeck commented Mar 14, 2014

  if asynchronicity is an optimization (that is, a complexity introduced for the sake
  of performance), why would people, a priori, turn to a framework that imposes it for
  everything? If asynchronous code is harder to reason about, why would we elect to
  live in a world where it is the default?

Node.js imposes asynchronous operation for nearly everything because nearly everything meaningful requires at least a few IO operations, and anywhere you have IO you can be doing other things while you wait for a response. Anywhere you write to disk, make a web request, send out a Redis command, execute a database query rather than passively waiting for a response to come back you could be doing something else instead.

In the case of a web server, you can start answering another request while waiting for a database query for your previous request to finish. That is the power of Node.js, and what allows it handle thousands of concurrent connections to get maximum requests per second served per machine.

 I basically need to draw a diagram of what happens

Give it a few months and you'll be naturally coding in async style with no need for diagrams. Perhaps use for the time being. It's really easy to understand because you are basically just defining data dependencies for each step and letting handle the flow control to ensure that each step has the data it needs when it gets executed.

Copy link

wtfil commented Mar 14, 2014

Look at this
It allow you to write non-blocking code in synchronicity style

Copy link

dch commented Mar 15, 2014

Erlang provides a far better set of primitives as a language for concurrent programming, without the headaches of shared state (except when you really need it). But it has a fraction of the packages that Node does, and doesn't have the same composability. I dream of a place somewhere between the two languages.

Copy link

Z3TA commented Dec 4, 2015

Asynchronous scripting takes some time to get used to, but when you get used to it, it's not any harder then reading any other language that has functions / subroutines / jumping to labels.

function cakeReady(cake) {
·· eat(cake);


But if you really want to read from line to line and not jump between functions, you can in-line the function:

bakeCake(function cakeReady(cake) {
·· eat(cake);

And some people like to add the word "then" to make it easier to follow (aka "promises")


You can also use the OO-pattern:

var cake = new Cake();
cake.ready = cakeReady;

And some people prefer:

cake.on("ready", cakeReady);

Copy link

solowt commented Jan 30, 2016

My feeling is that async programming with callbacks has a learning curve, but once you get it, then you get it. You won't generally have trouble with it again.

For me it took about a week of working on a series of nested for loops making asynchronous calls via github's api before it really sunk in. Yeah, it was frustrating working on that problem, but when it all came together, it made a lot of sense and that process probably made me a better programmer. Plus, that same series of calls done synchronously would have taken an unacceptably long amount of time (I was making between 10-300 calls). So if I was using ruby, I would have had to look into some kind of non-blocking technique in ruby (which I'm not familiar with).

Regarding promises, I think they're useful but overrated. They don't add functionality, they just make things prettier and easier for someone else to grasp what's going on in your code (promises are basically syntactic sugar for callbacks). Obviously those are both important things, but I think it's a mistake to use promises as an excuse to avoid learning asynchronous code flow. If you want to use node a lot, you're going to have to face asynchronous thinking eventually (or at least I did in my first week with node).

For example, I'd argue that Q.all([a,b]).then(c); IS much different than a(); b(); c();.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment