intercepting failed specs in jasmine (1 & 2)

jasmine-intercept.md

UPDATE 30 Oct 2013 ~ example here is now in its own repo as jasmine-intercept.

---

While developing a POC for a where() clause in the jasmine BDD framework, I found I couldn't use jasmine to spyOn() easily (i.e., without checking for the API differences between v1 and v2) in order to intercept expected failure messages in specs.

That's not a big deal when you're developing with tests at the laptop/workstation, but can be confusing to an continuous integration environment where failing specs halt the build. I wanted to assert that an expectation had failed, but without failing the whole spec itself.

Wrapping expectations in expect().toThrow() works for capturing expected errors as opposed to expected failures. When an expectation fails, jasmine will create an Error internally and use it for the content of the spec's result message. By that time, it's too late to prevent the result from being posted to the reporter as a failure.

The strategy exampled below shows one way to get around that, by overriding the addResult() (v1) and addExpectationResult() (v2) methods in order to capture the outgoing messages, and then restoring the original methods when resuming "real" expectations.

This may wind up in its own repo ( done -- see update at top and further examples below) or as an addition to jasmine's spy/mock/intercept capabilities (after much refactoring and negotiation). If you know another way to do this, please let me know so I can avoid continous bike-shedding, gold-plating, etc.

/*
 * Use these tests to intercept result messages set by jasmine, particularly on specs
 * that fail.
 */

describe('intercepting expected failing specs', function () {

  /*
   * set up vars for each iteration first
   */
   
  var currentSpec;
  var result;
  
  var passCount;
  var failMessages;

  var addResult;
  var addExpectationResult;
  var restore;
  
  
  beforeEach(function() {

    /* 
     *  Set up an interceptor for add-results methods.
     *  Call restore() to un-set these before expect() calls after the where clause.
     */
    
    currentSpec = jasmine.getEnv().currentSpec;
    
    result = /* jasmine 2.x.x. */ currentSpec.result || 
             /* jasmine 1.x.x. */ currentSpec.results_;
                 
    passCount = 0;
    failMessages = [];

    /* jasmine 1.x.x. */
    addResult = result.addResult;
    result.addResult = function (results) {
      if (results.trace) {
        failMessages.push(results.message);
      } else {
        passCount += 1;
        addResult.call(result, results);
      }
    }
    
    /* jasmine 2.x.x. */
    addExpectationResult = currentSpec.addExpectationResult;
    currentSpec.addExpectationResult = function (passed, data) {
      if (!passed) {
        failMessages.push(data.message);
      } else {
        passCount += 1;
        addExpectationResult.call(passed, data);
      }          
    };
    
    restore = function() {
      result.addResult = addResult;
      currentSpec.addExpectationResult = addExpectationResult;
    };
    
  });
  
  
  it('should return messages for incorrect expectation', function () {
  
    where(function(){/* 
        a  |  b  |  c
        1  |  1  |  1
        1  |  2  |  2
        4  |  2  |  4
        4  |  8  |  7
      */
      expect(Math.max(a, b)).toBe(Number(c));
    });
    
    /*
     * call restore here to turn off message interception
     */
    restore();
    
    expect(failMessages.length).toBe(1);
    expect(passCount).toBe(3);
    expect(failMessages[0]).toBe("Expected 8 to be 7.");
  });
  
});

30 Oct 13 - refactored to jasmine-intercept, usable as follows:

it('should work', function () {

  var a = 1, b = 2;

  intercept(); // turn it on
  
  expect(a).toBe(b); // this will be captured as a fail
  expect(a).toBe(a); // this will be captured as a pass
  
  intercept.clear(); // turn it off

  // now use real expectation to inspect messages

  expect(intercept.failMessages.length).toBe(1);
  expect(intercept.failMessages[0]).toBe('Expected ' + a + ' to be ' + b + '.');
  expect(intercept.passMessages.length).toBe(0);
});

31 Oct 13 - refactored to jasmine-intercept, usable as follows

better names? better api? second thoughts on the how-you-use-it part - maybe this would "look and feel" better?

it('iterates', function (done) {

  var a = 1, b = 2;
  
  var messages = intercept(function() {
    expect(a).toBe(b); // should fail
    expect(a).toBe(a); // should pass
  });
  
  expect(messages.failing.length).toBe(1);
  expect(messages.failing[0]).toBe('Expected ' + a + ' to be ' + b + '.');
  expect(messages.passing.length).toBe(1);
});

The first is mentally parsed easier, while the second provides a more satisfying cleanup. I think the second would be preferable.

Thinking of it in terms of CoffeeScript (since this'll inevitably be used in that context too):

it "iterates", (done) ->
  a = 1
  b = 2

  messages = intercept ->
    expect(a).toBe b # should fail
    expect(a).toBe a # should pass

  expect(messages.failing.length).toBe 1
  expect(messages.failing[0]).toBe "Expected " + a + " to be " + b + "."
  expect(messages.passing.length).toBe 1

It's pretty clean, really.

Just thinking out loud here. This is shorter but obviously less powerful.

it('iterates', function (done) {

  var a = 1, b = 2;

  intercept(function() {
    expect(a).toBe(b); // should fail
    expect(a).toBe(a); // should pass
  }).withPassing(1).withFailing(1);
});