Ember.js - Writing contract tests for your Ember Data models

_README.md

Ember.js Testing package was a great addition to the project. It allowed us to have fast specs to guarantee the defined behavior. However, there's no convention on how you should guarantee that your models, the heart of any Ember.js application, are valid in relation to your backend.

Put another way, if you have an User model with a name attribute, how can you be sure that your backend is still responding with {"user": {"name": "Your Name"}}? So, even though your acceptance specs pass, you're stubbing out your models with FIXTURES.

Most people either manually test their apps or create an end-to-end test with the backend server, which is slow as hell (think of Capybara). The good news is that there are conventions for solving this since like forever. One way to guarantee this integrity is via Contract tests (http://martinfowler.com/bliki/IntegrationContractTest.html). Basically, you have a test to guarantee your models are matching your backend.

Using server-side end-to-end tests have many drawbacks, such as unwanted coupling, slowness and, as a consequence, diverts the developer around testing edge-cases. On the social side, imagine if you were about to contribute to the Discourse open source project (http://www.discourse.org), but had to write its tests in Ruby (the project uses Rails framework). A great part of the audience that codes in other languages would be virtually blocked from contributing.

The following solution is based on three assumptions:

1. On your backend, whenever you need to communicate with a third party service (e.g Google Analytics), you don't write every test touching the external server. You stub the calls to make yours tests fast. Likewise, developers should test their Ember.js apps using App.User.FIXTURES without fear.
2. To solve the problem of having brittle tests that could give you false positives, you write what's called Contract Tests, tests responsible for touching the real external server to guarantee its interface is still the same. These are usually run only on the CI server.
3. During development, you don't use the real service, but a sandbox.

In that architecture, your Ember.js app is like your backend app and your backend is like an external service.

The solution

The proposed solution is this:

1 . you have an endpoint in your backend app that retrieves informations about your models or serializers. This is only accessible in your development environment. The format of the response is this:

"model_name": {
  "attributes": ["id", "total", "created_at", "environment"],
  "associations": ["items"]
}

2 . the Contract Test requests via Ajax the information about the model under tests. 3 . the Contract Test automatically checks the Ember Data model's attributes and matches.

The results is this:

In this case, we have many contract tests and one of them is failing: our FIXTURES are invalid. The results show us exactly what fields are missing. Although we're not testing that we can run CRUD operations on the backend, we're making sure the interfaces are respecting the established contracts. This would allow us to focus on the related areas of the app before going to production with buggy code.

Here's the code you'd write, which is part of the what's in the picture above:

var contract;

module("Contracts/Models/Order", {
  setup: function() {
    contract = new EmberTesting.modelContract({
      model: App.Order,
      root: "order",
      // endpoint available only under development
      contractUrl: "/admin/api/v1/resources?model=order"
    });
  }
});

asyncTest("obeys attributes contract", function() {
  contract.assertAttributes();
});

asyncTest("obeys relashionships contract", function() {
  contract.assertAssociations({
    // here, I'm experimenting with cart and don't want it
    // interfering in my specs just now
    except: ["cart"]
  });
});

asyncTest("it has valid fixtures", function() {
  contract.assertFixtures();
});

The lib can be found below.

Rails specific

This is what I use in my backend to retrieve the serializer information (I'm using only ActiveModel::Serializer at this moment).

class Api::ResourcesController < ApplicationController
  skip_before_filter :authenticate_admin_user!

  def show
    responder = const_get(params[:model])
    render json: {
      params[:model] => {
        attributes:   responder._attributes.keys,
        associations: responder._associations.keys
      }
    }
  end

  def const_get(model_name)
    camelized_model = model_name.camelize
    # If we're looking for User, we look for UserSerializer too.
    # In this example, I'm using only ActiveModel::Serializer, so it responds
    # to _attributes and _associations. I added below the model just to exemplify
    # how this code can be improved.
    [camelized_model, "#{camelized_model}Serializer"].inject do |memo, model|
      memo = Module.const_get(model) rescue nil
    end
  end
end

ember_contracts.js

var EmberTesting = {
  getStore: function(app) {
    return app.__container__.lookup("store:main");
  },

  modelContract: function(options) {
    this.model           = options.model,
    /**
     * Root element of the JSON response, if any
     */
    this.root            = options.root || null,
    this.contractUrl     = options.contractUrl;
    this.response        = null;

    this.assertAttributes = function() {
      var _this = this;
      this.assert(function(response) {
        QUnit.deepEqual(_this.localAttributes(), response.attributes);
      });
    }
    this.assertAssociations = function(options) {
      var _this  = this,
          except,
          localRelationships;

      if (options) {
        except = options.except;
      }

      /**
       * if an exception was passed in it, it means the contract should
       * disregard this exception.
       */
      var ApplyExceptions = function(except, array) {
        if (except) {
          except.forEach(function(exception) {
            var index = array.indexOf(exception);
            if (index >= 0)
              array = array.slice(index+1);
          })
        }
        return array;
      }

      localRelationships = ApplyExceptions(except, _this.localRelationships());

      this.assert(function(response) {
        associations = ApplyExceptions(except, response.associations);
        QUnit.deepEqual(localRelationships, associations);
      });
    }

    this.assertFixtures = function() {
      var _this = this;
      this.assert(function(response) {
        _this.model.FIXTURES.forEach(function(fixture) {
          var fields = Object.keys(fixture);
          QUnit.deepEqual(fields, response.attributes);
        });
      });
    }
    
    this.assert = function(assertion) {
      var _this = this;
      $.get(_this.contractUrl).done(function(response) {
        _this.response = response;

        if (_this.root) {
          _this.response = response[_this.root];
        }

        assertion(_this.response);
        start();
      });
    };

    this.localAttributes = function() {
      var _this = this,
          attributes = [];

      attributes = [];
      attributes.push("id");
      Ember.get(this.model, 'attributes').forEach(function(name, meta) {
        attributes.push(name);
      });
      return attributes;
    };

    this.localRelationships = function() {
      var _this = this,
          relationships = [];

      Ember.get(this.model, 'fields').forEach(function(name, kind) {
        if (kind.match("hasMany|belongsTo|hasOne")) {
          relationships.push(name);
        }
      });
      return relationships;
    };

    return this;
  }
}