public
Last active

Meteor Async Guide

  • Download Gist
meteor-async.md
Markdown

From Meteor's documentation:

In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node. We find the linear execution model a better fit for the typical server code in a Meteor application.

This guide serves as a mini-tour of tools, trix and patterns that can be used to run async code in Meteor.

Basic async

Sometimes we need to run async code in Meteor.methods. For this we create a Future to block until the async code has finished. This pattern can be seen all over Meteor's own codebase:

Meteor.methods({
  asyncJob: function(message) {

    // Set up a future
    var fut = new Future();

    // This should work for any async method
    setTimeout(function() {

      // Return the results
      fut.ret(message + " (delayed for 3 seconds)");

    }, 3 * 1000);

    // Wait for async to finish before returning
    // the result
    return fut.wait();
  }
});

Code

Parallel async

Sometimes we want to run more than one async methods in parallel and wait for them all to complete before returning:

Meteor.methods({
  parallelAsyncJob: function(message) {

    // We're going to make http get calls to each url
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    // Keep track of each job in an array
    var futures = _.map(urls, function(url) {

      // Set up a future for the current job
      var future = new Future();

      // A callback so the job can signal completion
      var onComplete = future.resolver();

      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Do whatever you need with the results here!

        // Inform the future that we're done with it
        onComplete(error, result);
      });

      // Return the future
      return future;
    });

    Future.wait(futures);
  }
});

Code

If you want to collect results from parallel async jobs you'll have to do a little more work:

Meteor.methods({
  parallelAsyncJob: function(message) {
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    var futures = _.map(urls, function(url) {
      var future = new Future();
      var onComplete = future.resolver();

      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Get the title if there was no error
        var title = (!error) && getTitle(result);

        onComplete(error, title);
      });

      return future;
    });

    // wait for all futures to finish
    Future.wait(futures);

    // and grab the results out.
    return _.invoke(futures, 'get'); 
  }
});

Code

Unblocking Meteor.methods

Each approach described so far breaks from Meteor's default synchronous programming style, but requests from a given client are still processed one at a time. If you have a method that could take a long time call this.unblock() and subsequent requests will be run in a new fiber.

Meteor.methods({
  moveForkliftToBuilding9: function(message) {
    this.unblock();
    forklift.moveTo({ building: 9 })
  }
});

Note: borrowed this method name from Matt Debergalis because I think it's hilarious

Accessing Meteor's full environment from async callbacks

If you need to access Meteor's full environment in an asynchronous method's callback Meteor provides a wrapper called Meteor.bindEnvironment:

TODO make better example

var fn = function(error, result) {
  console.log('done ' + result);
};

fn = Meteor.bindEnvironment(callback, function(e) {
  var message = "Something went wrong! " +
                "Everything is broken! " + 
                "Your life is ruined!";

  console.log(message, e.stack);
});

myAsyncMethodThatNeedAccessToMeteorEnv(fn);

Packaging async node modules

TODO

Abstractions

At this point I'm leaving this purposely unfinished because until I have opportunities to use these techniques more than once, in real apps, I don't want to guess what the right abstractions are. Hopefully people will write awesome smart packages for this stuff and we can find out together what will be most effective and then campaign for our favorites to be be included in core.

TODO

Include all useful info mentioned by Matt here

Instead of example like asyncJob and parallelAsyncJob come up with meaningful names and implementations

Contributions

Thanks to the following people for their help with this:

Tom Coleman @tmeasday

Resources

http://stackoverflow.com/a/11510874

This is great. Any chance of this being documented?

The Future class seem to have been removed in Meteor 0.5.1 - any chance to get this back with a simple require?

require('fibers/future') is enough. I added a little howto here: https://gist.github.com/4130605

I think it would be nice to include the client side call as well for full working example - I had the client side callback arguments wrong for a while and it took me a while to spot it. Something like this should work:

Template.hello.events({

  'click input' : function () {

    //call the async function from the client
    Meteor.call("asyncJob", "async test", function(err, res){

      alert(res);

    });
  }
});

In Meteor 0.6.0, add:

var Future = Npm.require("fibers/future")

to the top of your file.

I made also a simple library to wrap common async functions into blocking ones.

Helpfull, thank you!

Just a heads up though, as of 5 days ago (Meteor 0.6.5) future.ret() is deprecated. You use it in your "Basic async" example.

I found out about it here: https://github.com/meteor/meteor/issues/1311

Is there any async in meteor's roadmap?

Yes, if you need blocking methods for async calls you can use Async Utilities from npm smart package. If you need non-blocking methods you can use Q.js and promises. See my gist: https://gist.github.com/hipertracker/8064847

Please sign in to comment on this gist.

Something went wrong with that request. Please try again.