Multiple async tests against multiple objects with any JS test runner.

gistfile1.js

"use strict";

/*global process, console */

function Thingy() {}
Thingy.prototype.returns5 = function(cb) { process.nextTick(function() { cb(null, 5); }); };
Thingy.prototype.returns8 = function(cb) { process.nextTick(function() { cb(null, 8); }); };

var thingy1 = new Thingy(),
    thingy2 = new Thingy(),
    thingy3 = new Thingy(),
    thingy4 = new Thingy();

/*
OK. We have four objects, thingy1-4. We want to test each of those objects
to confirm that (a) their returns5() method returns 5 (this is "returns"
by calling a callback; i.e., it's async) and that (b) their returns8() method
returns 8.
So, how?

Note that the test output ought to look like this:

Testing thingy1's returns5 function: pass
Testing thingy2's returns5 function: pass
Testing thingy3's returns5 function: pass
Testing thingy4's returns5 function: pass
Testing thingy1's returns8 function: pass
Testing thingy2's returns8 function: pass
Testing thingy3's returns8 function: pass
Testing thingy4's returns8 function: pass

i.e., each thing is a separate test.

But there should only be one function which tests whether 
someobject.returns5() actually returns 5; not one function per item.
*/

Edit: this is rubbish, trying again.

I think I'd generate these tests – but I doubt all frameworks are happy with you doing that!

it.should('return five in all cases', function (done) {
  async.map(
    thingies,
    function (thingy, cb) { return thingy.returns5(cb) },
    function (err, results) {
      // check results are all 5
      done();
    }
  )
});

Most likely that could be generalised into a nice helper.

Nuts/helpful/wrong/other?

That doesn't check each one. It checks them all and then says "one didn't pass" :( The test runner should be running each one as a test and checking the results itself; what you're doing is running all the tests yourself and then checking all the results yourself. That's the test runner's job, not yours :(

I mean, imagine that you wanted to test two properties on an object. Would you do

equal(obj.prop1, "val1");
equal(obj.prop2, "val2");

or would you do

equal([obj.prop1, obj.prop2], ["val1", "val2"]) ?

Because the second one is (a) stupid and (b) what you're doing above. It's what I thought of too (and it doesn't really apply to things more complex than this little toy example), but I hate it.

What I mean by the "toy example" comment is that I don't really want a test that tests (all things).returns5(), I want a test that tests object.(all methods) so that thingy1.returns5 and thingy1.returns8 are tested together, then thingy2.returns5 and thingy2.returns8, and so on. Imagine if a thingy were expensive to create, for example.

With tap:

var t = require('tap');

t.test('thingies', function (t) {

  async.each(
    thingies,
    function (thingy) {
        t.test('returns 5', function (t) {
            thingy.returns5(function (err, res) {
                t.pass(res === 5);
                t.end();
            });
        });
    },
    t.end // might need to be bound
  )

});

Totally untested, of course.

I don't follow why equal([obj.prop1, obj.prop2], ["val1", "val2"]) is stupid – it's the same as the other example, expect that order matters, but leaving that aside...

Why is that you need to test multiple object with the same methods? When will they be different?

The objects are essentially database models. The tests are things like "does this object actually save correctly to the DB when I call object.save()?", "does its last-updated-date get updated when I call object.save()?", that sort of thing. So I have many different models, and I want to confirm certain basic functionality on all of them. Then, separately, I have a set of things which pretend to be database models but are actually wrappers around an API, and I want those to pass all the same tests to confirm that they correctly act like DB models even though they aren't.

On the equal() thing,

equal(obj.dinner, "fish", "Dinner should be fish");
equal(obj.drink, "lager", "Lager goes well with fish");

is much nicer than

equal([obj.dinner, obj.drink], ["fish", "lager"], "Something is wrong, but I'm not going to tell you what");

In reverse order:

Indeed, but any good assertion lib will say "got [x, y], expected [a, b]" and if those are related (and they must be, they're both being tested) then maybe it's not so bad. But neither of us would actually do that so it's moot.

To the testing thing, I suspect I'd write a set of helpers, so that my test code looked like:

var t = require('tap');

makeTestsFor('thingies', thingies, function (thingy, t) {
    t.test(thingy.name + ' returns 5', function (t) {
        thingy.returns5(function (err, res) {
            t.pass(res === 5);
            t.end();
        });
    });

    t.test(thingy.name + ' returns 8', function (t) {
        thingy.returns8(function (err, res) {
            t.pass(res === 8);
            t.end();
        });
    });

    t.end(); // not sure about the async properties of this
});

(edited to make betterer)