Skip to content

Instantly share code, notes, and snippets.

@Raynos Raynos/x.md Secret
Created Aug 5, 2012

Embed
What would you like to do?

Taming asynchronous code through functional programming

There are surprisingly many parallels between managing callback hell and functional programming. This article will lead you through common pitfalls involved with writing asynchronous JavaScript and demonstrate how functional programming techniques make these pitfalls disappear.

We shall investigate an example of messy code and see existing control flow solutions that clean up the messy code. We will then look into partial function application and function composition and see how those techniques apply to clean up the messy code

Callback hell

Callback hell is the phenomenon where you end up with a lot of nested functions and a high level of indentation. This commonly happens when you use a lot of in-line function expressions. Callback hell is common if you're writing code with node.js which generally involves a lot of asynchronous callbacks or if you're using something in the browser and using nested event handlers and ajax calls.

Let's take an example from callbackhell.com

getThingFromHardDrive(function(thing) {
    uploadToServer(thing, function(response) {
        saveToLogDatabase(response, function(logResponse) {
            getAnotherThingFromHardDrive(function(thing) {
                // etc
            })
        })
    })
})

The code above is actually a list of asynchronous actions that need to happen in the specific sequence. You could be tempted to refactor the code using a library like seq to represent it as a sequence of actions

Seq()
    .seq(function () {
        getThingFromHardDrive(this)
    })
    .seq(function (thing) {
        uploadToServer(thing, this)
    })
    .seq(function (response) {
        saveToLogDatabase(response, this)
    })
    .seq(function (logResponse) {
        getAnotherThingFromHardDrive(this)
    })
    .seq(function (thing) {
        // etc
    })

Alternatively you could recognize that it's actually a waterfall of actions that need to occur. Then you would use a library like async to represent the code as the flow of a waterfall

async.waterfall([
    function (callback) {
        getThingFromHardDrive(callback)
    }
    , function (thing, callback) {
        uploadToServer(thing, callback)
    }
    , function (response, callback) {
        saveToLogDatabase(response, callback)
    }
    , function (logResponse, callback) {
        getAnotherThingFromHardDrive(callback)
    }
], function (err, thing) {
    // etc
})

Or taking advantage of the function signature match

async.waterfall([
    getThingFromHardDrive
    , uploadToServer
    , saveToLogDatabase
    , function (logResponse, callback) {
        getAnotherThingFromHardDrive(callback)
    }    
], function (err, thing) {
    // etc
})

Both of these solutions represent the original code as something that doesn't suffer from callback hell. The main advantage here is the lack of indentation and the more linear appearance of the code. In general using a control flow library like seq or async makes the code more maintainable.

However, the issue with these libraries is that they hurt readability unless you're familiar with them. Specifically the concept of a sequence or waterfall does not come naturally to JavaScript. Another issue is that these types of control flow libraries make it to easy to turn your code into something that looks like a DSL (There are actually real DSLs for managing control flow like streamline)

Functional programming

Now you might ask the question, where does functional programming come into this? I hear your sentiment, let's take a look at the notion of partial function applications. A partially applied function is a function which has some arguments frozen in place. We can use ap to demonstrate this using examples

function sum(a, b) {
    return a + b
}

var addFive = ap.partial(sum, 5)
    , fifteen = addFive(10)

addFive is a partial application of sum. It's a version of the sum function which has the first argument frozen as the value 5.

Then when we call addFive(10) it's actually equivalent to calling sum(5, 10) so returns 15

We can use partial function applications to simplify the example we had.

getThingFromHardDrive(onHardDriveSave)

function onHardDriveSave(thing) {
    uploadToServer(thing, onUpload)
}

function onUpload(response) {
    saveToLogDatabase(response, onLogSave)
}

function onLogSave(logResponse) {
    getAnotherThingFromHardDrive(finish)
}

function finish(thing) {
    // etc
}

When trying to apply partial function applications to our example we actually see they are not necessary when we use function declarations. You can also see that the function declarations are in order. It turns out that the seq, waterfall and partial application abstractions are not needed when we express the coded as function declarations. Let's take a look at an example which does actually need partial function applications to work (without error handling code for brevity)

database.open(function (err, client) {
    client.collection("myCollection", function (err, collection) {
        collection.insert({
            hello: "world"
        }, {
            safe: true
        }, function (err, data) {
            client.close()
        })
    })
})

The above example has the client variable used in the last function which isn't a parameter. So we can't just split this out into function declarations and have it just work. However we can use partial applications

database.open(openClientCollection)

function openClientCollection(err, client) {
    client.collection("myCollection", ap.partial(insertData, client))
}

function insertData(client, err, collection) {
    collection.insert({
        hello: "world"
    }, {
        safe: true
    }, ap.partial(closeClient, client))
}

function closeClient(client, err, data)  {
    client.close()
}

What we are doing here is effectively taking variables that we used to reference through closures and freezing them in the parameter list of a function. In the above example we pass the client variable through two functions.

Function composition

However we can do better then this when we recognize that the above code is actually a composition of numerous functions in an asynchronous manner. Let's look at a simple example of function composition using composite

function addFive(n) {
    return n + 5
}

function multiplyByTwo(n) {
    return n * 2
}

function square(n) {
    return n * n
}

var squareThenAddFiveThenMultiplyByTwo = 
    compose(multiplyByTwo, addFive, square)

var eightTeen = squareThenAddFiveThenMultiplyByTwo(2)

Composition takes a number of functions and returns a new function that applies each function onto the input in order. If we redefine the return value of a function as calling the last argument (the callback function) with (err, result) we can actually generically compose asynchronous functions together

var doAction = composeAsync(
    closeClient
    , insertData
    , openClientCollection
    , database.open.bind(database)
)

doAction.call({})

function openClientCollection(err, client, callback) {
    this.client = client
    client.collection("myCollection", callback)
}

function insertData(err, collection, callback) {
    collection.insert({
        hello: "world"
    }, {
        safe: true
    }, callback)
}

function closeClient(err, data)  {
    this.client.close()
}

What we've done above is described the action we want to take as a composition of the four individual function calls. We have also called the function with a specific thisValue which is used in all calls. This allows us to store temporary values on the thisValue.

Although using asynchronous function composition might look similar to using seq or waterfall, the difference is that we are using a well known functional programming technique to reduce the complexity of our code instead of bringing in a library whose specific purpose is "control flow".

There are other functional programming techniques like map, filter and reduce that we can also use to make asynchronous programming easier. We will look at these in the future.

@jcolebrand

This comment has been minimized.

Copy link

jcolebrand commented Aug 5, 2012

What we've done above is described the action we want to take as a composition of the four individual function calls. We have also called the function with a specific thisValue which is used in all calls. This allows us to store temporary values on the thisValue.

It looks like you're saying once in the middle "don't do this" and then at the end "do this after all" but I see the point.

Although using asynchronous function composition might look similar to using seq or waterfall, the difference is that we are using a well known functional programming technique to reduce the complexity of our code instead of bringing in a library who's specific purpose is "control flow"

"whose specific purpose" not "who is specific purpose"

@jcolebrand

This comment has been minimized.

Copy link

jcolebrand commented Aug 5, 2012

Also, that last paragraph seems to be leaving something off ... not sure what exactly. Like you trail off to nowhere. Maybe put a link to the second article?

@Raynos

This comment has been minimized.

Copy link
Owner Author

Raynos commented Aug 5, 2012

@jcolebrand

It looks like you're saying once in the middle "don't do this" and then at the end "do this after all" but I see the point.

I don't understand what you mean

@jcolebrand

This comment has been minimized.

Copy link

jcolebrand commented Aug 5, 2012

However, the issue with these libraries is that they hurt readability unless you're familiar with them. Specifically the concept of a sequence or waterfall does not come naturally to JavaScript.

@Raynos

This comment has been minimized.

Copy link
Owner Author

Raynos commented Aug 5, 2012

@jcolebrand I'm implying that the notion of a sequence or waterfall is not natural. But asynchronous function composition is natural because it's just function composition which is a standard FP pattern and JavaScript is functional.

It's opinionated and hard to get across

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.