Create a gist now

Instantly share code, notes, and snippets.

What would you like to do?
Meteor Async Guide

From Meteor's documentation:

In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node. We find the linear execution model a better fit for the typical server code in a Meteor application.

This guide serves as a mini-tour of tools, trix and patterns that can be used to run async code in Meteor.

Basic async

Sometimes we need to run async code in Meteor.methods. For this we create a Future to block until the async code has finished. This pattern can be seen all over Meteor's own codebase:

Meteor.methods({
  asyncJob: function(message) {
  
    // Set up a future
    var fut = new Future();

    // This should work for any async method
    setTimeout(function() {

      // Return the results
      fut.ret(message + " (delayed for 3 seconds)");

    }, 3 * 1000);

    // Wait for async to finish before returning
    // the result
    return fut.wait();
  }
});

Code

Parallel async

Sometimes we want to run more than one async methods in parallel and wait for them all to complete before returning:

Meteor.methods({
  parallelAsyncJob: function(message) {

    // We're going to make http get calls to each url
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    // Keep track of each job in an array
    var futures = _.map(urls, function(url) {

      // Set up a future for the current job
      var future = new Future();

      // A callback so the job can signal completion
      var onComplete = future.resolver();

      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Do whatever you need with the results here!
    
        // Inform the future that we're done with it
        onComplete(error, result);
      });

      // Return the future
      return future;
    });

    Future.wait(futures);
  }
});

Code

If you want to collect results from parallel async jobs you'll have to do a little more work:

Meteor.methods({
  parallelAsyncJob: function(message) {
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    var futures = _.map(urls, function(url) {
      var future = new Future();
      var onComplete = future.resolver();
  
      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Get the title if there was no error
        var title = (!error) && getTitle(result);
    
        onComplete(error, title);
      });
  
      return future;
    });

    // wait for all futures to finish
    Future.wait(futures);

    // and grab the results out.
    return _.invoke(futures, 'get'); 
  }
});

Code

Unblocking Meteor.methods

Each approach described so far breaks from Meteor's default synchronous programming style, but requests from a given client are still processed one at a time. If you have a method that could take a long time call this.unblock() and subsequent requests will be run in a new fiber.

Meteor.methods({
  moveForkliftToBuilding9: function(message) {
    this.unblock();
    forklift.moveTo({ building: 9 })
  }
});

Note: borrowed this method name from Matt Debergalis because I think it's hilarious

Accessing Meteor's full environment from async callbacks

If you need to access Meteor's full environment in an asynchronous method's callback Meteor provides a wrapper called Meteor.bindEnvironment:

TODO make better example

var fn = function(error, result) {
  console.log('done ' + result);
};

fn = Meteor.bindEnvironment(callback, function(e) {
  var message = "Something went wrong! " +
                "Everything is broken! " + 
                "Your life is ruined!";

  console.log(message, e.stack);
});

myAsyncMethodThatNeedAccessToMeteorEnv(fn);

Packaging async node modules

TODO

Abstractions

At this point I'm leaving this purposely unfinished because until I have opportunities to use these techniques more than once, in real apps, I don't want to guess what the right abstractions are. Hopefully people will write awesome smart packages for this stuff and we can find out together what will be most effective and then campaign for our favorites to be be included in core.

TODO

Include all useful info mentioned by Matt here

Instead of example like asyncJob and parallelAsyncJob come up with meaningful names and implementations

Contributions

Thanks to the following people for their help with this:

Tom Coleman @tmeasday

Resources

http://stackoverflow.com/a/11510874

@punund

This comment has been minimized.

Show comment Hide comment
@punund

punund Oct 5, 2012

This is great. Any chance of this being documented?

punund commented Oct 5, 2012

This is great. Any chance of this being documented?

@joscha

This comment has been minimized.

Show comment Hide comment
@joscha

joscha Nov 22, 2012

The Future class seem to have been removed in Meteor 0.5.1 - any chance to get this back with a simple require?

joscha commented Nov 22, 2012

The Future class seem to have been removed in Meteor 0.5.1 - any chance to get this back with a simple require?

@joscha

This comment has been minimized.

Show comment Hide comment
@joscha

joscha Nov 22, 2012

require('fibers/future') is enough. I added a little howto here: https://gist.github.com/4130605

joscha commented Nov 22, 2012

require('fibers/future') is enough. I added a little howto here: https://gist.github.com/4130605

@semateos

This comment has been minimized.

Show comment Hide comment
@semateos

semateos Feb 27, 2013

I think it would be nice to include the client side call as well for full working example - I had the client side callback arguments wrong for a while and it took me a while to spot it. Something like this should work:

Template.hello.events({

  'click input' : function () {

    //call the async function from the client
    Meteor.call("asyncJob", "async test", function(err, res){

      alert(res);

    });
  }
});

I think it would be nice to include the client side call as well for full working example - I had the client side callback arguments wrong for a while and it took me a while to spot it. Something like this should work:

Template.hello.events({

  'click input' : function () {

    //call the async function from the client
    Meteor.call("asyncJob", "async test", function(err, res){

      alert(res);

    });
  }
});
@avital

This comment has been minimized.

Show comment Hide comment
@avital

avital Apr 25, 2013

In Meteor 0.6.0, add:

var Future = Npm.require("fibers/future")

to the top of your file.

avital commented Apr 25, 2013

In Meteor 0.6.0, add:

var Future = Npm.require("fibers/future")

to the top of your file.

@mitar

This comment has been minimized.

Show comment Hide comment
@mitar

mitar Jun 28, 2013

I made also a simple library to wrap common async functions into blocking ones.

mitar commented Jun 28, 2013

I made also a simple library to wrap common async functions into blocking ones.

@jeroentbt

This comment has been minimized.

Show comment Hide comment
@jeroentbt

jeroentbt Aug 20, 2013

Helpfull, thank you!

Just a heads up though, as of 5 days ago (Meteor 0.6.5) future.ret() is deprecated. You use it in your "Basic async" example.

I found out about it here: meteor/meteor#1311

Helpfull, thank you!

Just a heads up though, as of 5 days ago (Meteor 0.6.5) future.ret() is deprecated. You use it in your "Basic async" example.

I found out about it here: meteor/meteor#1311

@johntday

This comment has been minimized.

Show comment Hide comment
@johntday

johntday Sep 30, 2013

Is there any async in meteor's roadmap?

Is there any async in meteor's roadmap?

@hipertracker

This comment has been minimized.

Show comment Hide comment
@hipertracker

hipertracker Dec 22, 2013

Yes, if you need blocking methods for async calls you can use Async Utilities from npm smart package. If you need non-blocking methods you can use Q.js and promises. See my gist: https://gist.github.com/hipertracker/8064847

Yes, if you need blocking methods for async calls you can use Async Utilities from npm smart package. If you need non-blocking methods you can use Q.js and promises. See my gist: https://gist.github.com/hipertracker/8064847

@skozz

This comment has been minimized.

Show comment Hide comment
@skozz

skozz Jul 14, 2014

This gist is awesome but bro it's deprecated.

Stackoverflow topic for Meteor 0.8+ http://stackoverflow.com/questions/24743402/how-to-get-an-async-data-in-a-function-with-meteor

skozz commented Jul 14, 2014

This gist is awesome but bro it's deprecated.

Stackoverflow topic for Meteor 0.8+ http://stackoverflow.com/questions/24743402/how-to-get-an-async-data-in-a-function-with-meteor

@Overload119

This comment has been minimized.

Show comment Hide comment
@Overload119

Overload119 Nov 22, 2014

+1 On the updated documentation.

Here's an example of how I'm using this with GeoNear that isn't supported using just Meteor yet.

Meteor.methods({
  searchQuery: function(searchParams, callback) {
    // Look through skills and work experience.
    // Sort by distance (using $near forces this).
    // Limit of 100 is also enforced by $near.
    // TODO Incorporate recently active
    check(searchParams, Object);

    if (!this.userId) {
      throw new Meteor.Error('User is not logged in');
    }

    if (!searchParams.term) {
      throw new Meteor.Error('Search parameters did not contain `term`.');
    }

    var asyncDbLookup = function(callback) {
      _db.command({
        geoNear: 'users',
        near: [searchParams.lat, searchParams.lng],
        limit: 30,
        query: {
          $or: [
            { interests: searchParams.term },
            { skills: searchParams.term },
            { jobExperience: searchParams.term }
          ]
        }
      }, function(err, res) {
        var results = [];

        // Pluck only the fields we want.
        if (res.results && res.results.length > 0) {
          results = _.map(res.results, function(entry) {
            var mappedEntry = _.pick(entry.obj, '_id', 'pictureUrl', 'headline', 'firstName');
            mappedEntry.distance = entry.dis;

            return mappedEntry
          });
        }

        callback(null, results);
      });
    };

    var syncDbLookup = Meteor.wrapAsync(asyncDbLookup);
    return syncDbLookup();
  }
});

+1 On the updated documentation.

Here's an example of how I'm using this with GeoNear that isn't supported using just Meteor yet.

Meteor.methods({
  searchQuery: function(searchParams, callback) {
    // Look through skills and work experience.
    // Sort by distance (using $near forces this).
    // Limit of 100 is also enforced by $near.
    // TODO Incorporate recently active
    check(searchParams, Object);

    if (!this.userId) {
      throw new Meteor.Error('User is not logged in');
    }

    if (!searchParams.term) {
      throw new Meteor.Error('Search parameters did not contain `term`.');
    }

    var asyncDbLookup = function(callback) {
      _db.command({
        geoNear: 'users',
        near: [searchParams.lat, searchParams.lng],
        limit: 30,
        query: {
          $or: [
            { interests: searchParams.term },
            { skills: searchParams.term },
            { jobExperience: searchParams.term }
          ]
        }
      }, function(err, res) {
        var results = [];

        // Pluck only the fields we want.
        if (res.results && res.results.length > 0) {
          results = _.map(res.results, function(entry) {
            var mappedEntry = _.pick(entry.obj, '_id', 'pictureUrl', 'headline', 'firstName');
            mappedEntry.distance = entry.dis;

            return mappedEntry
          });
        }

        callback(null, results);
      });
    };

    var syncDbLookup = Meteor.wrapAsync(asyncDbLookup);
    return syncDbLookup();
  }
});
@dennisharrison

This comment has been minimized.

Show comment Hide comment
@dennisharrison

dennisharrison Apr 7, 2015

@Overload119 - excellent example, thank you!

@Overload119 - excellent example, thank you!

@Vingdc

This comment has been minimized.

Show comment Hide comment
@Vingdc

Vingdc May 12, 2015

@Overload119 Thank you very much! finally understood how Async works on Meteor!

Vingdc commented May 12, 2015

@Overload119 Thank you very much! finally understood how Async works on Meteor!

@kublermdk

This comment has been minimized.

Show comment Hide comment
@kublermdk

kublermdk Nov 23, 2015

The rest of the Javascript world use Promises which is effectively what you have here.
Thank you for bringing this to Meteor.

The rest of the Javascript world use Promises which is effectively what you have here.
Thank you for bringing this to Meteor.

@marxo

This comment has been minimized.

Show comment Hide comment
@marxo

marxo Dec 24, 2015

Is there a fresh up-to-date fork of this or is this still considered valid?

marxo commented Dec 24, 2015

Is there a fresh up-to-date fork of this or is this still considered valid?

@thiagodelgado111

This comment has been minimized.

Show comment Hide comment
@thiagodelgado111

thiagodelgado111 Jul 12, 2016

Hey, great article! Can you put up a few samples of how to use async/await syntax in Meteor server methods definition? :)

Hey, great article! Can you put up a few samples of how to use async/await syntax in Meteor server methods definition? :)

@Dartv

This comment has been minimized.

Show comment Hide comment
@Dartv

Dartv Jul 20, 2016

+1 for async/await

Dartv commented Jul 20, 2016

+1 for async/await

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment