Acceptance Testing Data Collection: Asserting On Outbound Resource Requests Using CasperJS and PhantomJS
At Intent Media, we run ad networks, which means that on the data science team, we don't just science data, we also collect it.
One way in which we accomplish this collection is through the use of beacons on our partners' sites. And because we are nearly as excited about software testing as we are about statistical modeling and Hadoop-ing, we also want to do proper acceptance testing of our beacons.
Acceptance testing for us means that we need a way to assert that a browser, upon executing our javascript, will make a request to our beacon url. It turns out that our pal PhantomJS has a handy method for handling this kind of thing, and our favorite PhantomJS wrapper CasperJS emits an event on resource requests to help us out.
Armed with these tools, we can set up a listener on the
'resource.requested' event and
do something useful when it happens:
casper.onResourceRequestFor = function(requestRegexp, callback) {
var resourceListener = function(request) {
if (request.url.match(requestRegexp)) {
callback();
}
};
this.on('resource.requested', resourceListener);
};
Now we can require our little library in our test, which we'll write
in CoffeeScript:
require('./casper.intentmedia.js')
And we can make some assertions about requested resources:
test_page = 'http://localhost/test_page.html'
beacon_url = /http:\/\/localhost\/beacon\.html.*/
casper.test.begin "Page fires a beacon request", 1, (test) ->
casper.start 'http://localhost', ->
casper.open test_page
casper.then ->
casper.onResourceRequestFor beacon_url, ->
test.assert(true, 'Matching request made')
casper.run ->
test.done()
Note that because we rely here on CasperJS's tester module, we must invoke our test using the test subcommand: casperjs test beacon_test.coffee.
That test is almost useful, but if you're following along carefully you'll have noticed a problem. This test is flaky -- like croissant flaky.
The open method waits for the page load to finish, so we won't call the next navigation step on the stack (provided as a callback to then()) until after page load has finished.
Thus, when we make the assertion above, the browser may or may not have already executed the javascript that makes our beacon request.
We really need to set up our listener before the navigation step causing the event to fire.
casper.assertResourceRequestAfterStep = function(urlRegexp, step) {
this.onResourceRequestFor(urlRegexp) {
this.test.assert(true, 'matching request made');
}
step();
};
And then we reflect this change in our test script:
casper.assertResourceRequestAfterStep beacon_url, ->
casper.open test_page
Relying on a step timeout to fail this test means that we aren't providing super-useful feedback upon test failure. We'll add an explicit wait (using waitFor, documented here), and we'll add better failure and success messages.
casper.assertResourceRequestAfterStep = function(urlRegexp, step) {
var testStatus = 'fail';
var passTest = function() {
casper.test.assert(true, "Request made for #{urlRegexp}.");
};
var failTest = function() {
casper.test.assert(false, "No request made for #{urlRegexp}.");
};
this.onResourceRequestFor(urlRegexp) {
test_status = 'pass';
}
step();
this.waitFor(function() { return testStatus == 'pass' }, passTest, failTest);
};
Running our test now produces more helpful logging:
Test file: beacon_test.coffee
# Page fires a beacon request
PASS Request made for /http:\/\/localhost\/beacon\.html.*/
PASS 1 test executed in 0.225s, 1 passed, 0 failed, 0 dubious, 0 skipped.
We need to do one other bit of housekeeping. If we don't explicitly remove the event listener we set up, it will persist through the suite. That might cause the suite to behave unpredictably, which in turn might cause us to owe our colleagues beers.
In this case, we're happy to remove the listener once we have seen a matching request:
casper.onResourceRequestFor = function(requestRegexp, callback) {
var removeResourceListener = function () {
casper.removeListener('resource.requested', resourceListener);
};
var resourceListener = function(request) {
if (request.url.match(requestRegexp)) {
removeResourceListener();
callback();
}
};
this.on('resource.requested', resourceListener);
};
Here's what our custom assertion library looks like now:
casper.onResourceRequestFor = function(requestRegexp, callback) {
var removeResourceListener = function () {
casper.removeListener('resource.requested', resourceListener);
};
var resourceListener = function(request) {
if (request.url.match(requestRegexp)) {
removeResourceListener();
callback();
}
};
this.on('resource.requested', resourceListener);
};
casper.assertResourceRequestAfterStep = function(urlRegexp, step) {
var testStatus = 'fail';
var passTest = function() {
casper.test.assert(true, "Request made for " + urlRegexp);
};
var failTest = function() {
casper.test.assert(false, "No request made for " + urlRegexp);
};
this.onResourceRequestFor(urlRegexp, function() {
testStatus = 'pass';
});
step();
casper.waitFor(function() { return testStatus == 'pass'; }, passTest, failTest);
};
And our test script:
require('./casper.intentmedia.js')
test_page = 'http://localhost/test_page.html'
beacon_url = /http:\/\/localhost\/beacon\.html.*/
casper.test.begin "Page fires a beacon request", 1, (test) ->
casper.start 'http://localhost', ->
casper.assertResourceRequestAfterStep beacon_url, ->
casper.open test_page
casper.run ->
test.done()
Now that's a flake-free and readable acceptance test.
PhantomJS here provides the ability to inspect the outgoing requests that represent the basic acceptance criterion for our beacon tag. And CasperJS’s event system makes the asynchronous nature of these requests manageable.
Anybody out there with a more elegant solution? Other handy uses for CasperJS? If so let us know, or better yet come work here!
"MapReducing" sounds--odd. Maybe that's intentional but I wonder if it can be sound a little less like we arbitrarily love processing data using a very specific distributed model.