Writing tests for node-mapnik

notes.md

Writing tests for node-mapnik

node-mapnik is a Node.js module that binds the functionality of Mapnik to be used via a a JavaScript API.

They key parts

• Mocha.js is our javascript testing suite that uses a set of functions and "assertions" to ensure the results of our code are as expected.
• coveralls.io is a great tool that we use to check for coverage of our tests. It runs our tests for us, and is able to see which lines of code are used, and which are not during those tests. If a particular line of code isn't used during the tests, it shows as red, allowing us to write a particular test for that scenario.
• The /test directory within the repository is where all of our tests are located.
• make test is the command that runs our tests. This is actually running mocha -R spec

What does a test look like?

Tests are structured into their own test.js files that mimick the .cpp files from the Mapnik source in order to keep things as clean as possible. Assume you are writing a test that covers a particular part of the code within /src/mapnik_vector_tile.cpp, you'll likely want to add a test to /test/vector-tile.test.js. This isn't 100% true all the time, but for the purposes of this writeup we can assume that.

A test in node-mapnik tries to check things like:

• if a value is of the proper type
• if errors are thrown at the appropriate time
• if input values match output values after running them through a function

These are very general statements, but comprise a majority of the tests we have. Say you want to write a test for a function mapnik.VectorTile to ensure none of the x, y, or z values are negative. We'll have to do a couple of things:

1. write a description for the test using it('description', function(done) { ... })
2. Create an instance of mapnik.VectorTile() within the test
3. assert that errors are thrown with negative values using assert.throws()
4. conclude the test with done() (which is a particular method of Mocha.js)

The test looks like this:

it('throws an error when x, y, or z values are negative', function(done) {
  // assert.throws is a check that ensures this case throws an error in your code
  // it requires an anonymous function to execute your code within the statement
  assert.throws(function() { new mapnik.VectorTile(-1,0,0); });
  
  // concludes the test
  done(); 
});

That covers if x is -1, but what about y, or z values? We can add two more tests to the code:

it('throws an error when x, y, or z values are negative', function(done) {
  assert.throws(function() { new mapnik.VectorTile(-1,0,0); }); // negative x
  assert.throws(function() { new mapnik.VectorTile(0,-1,0); }); // negative y
  assert.throws(function() { new mapnik.VectorTile(0,0,-1); }); // negative z
  done();
});

Now we have a test! You can run that with make test and your output should show a positive checkmark since in our mapnik_vector_tile.cpp code we throw errors for these negative values.

Adding coverage to tests

Sometimes tests are covering nearly all of the code in a file, which can be viewed through coveralls.io, but relying on Coveralls to continually update itself can take time, especially when you are trying to reach new places in the code. For example, in Coveralls we can see that there is a portion of our code within the addImageBuffer function that should throw when the first argument is not a buffer. Coveralls shows this area as red.

The above means none of our tests are actually running those lines of code, which means we aren't ensuring that error is thrown. In order to dig down into what we need, it's important to be able to log if our code actually hits that point.

Enter std::cout C++ print statements

Since Coveralls can take a while to run, a quick way locally check if your code is running on a particular line is to add print statements. In C++ this is a std::cout statement. In order to write a test that covers the lines above, let's add a print statement above:

if (info.Length() < 1 || !info[0]->IsObject())
{
  std::cout << "THIS IS THE ERROR WHERE THE FIRST ARGUMENT IS NOT A BUFFER OBJECT";        
  Nan::ThrowTypeError("first argument must be a buffer object");
  return;
}

Run make again to rebuild your code. Since this code is within mapnik_vector_tile.cpp, we'll want to add a new test to vector-tile.test.js just like above. NOTE: There will likely be a number of tests for each function already, so it's good to check if your test can be added to a pre-existing it() statement. For now, we'll assume there isn't one to keep things clean.*

Right now when we run make test, we won't see the console logged with that statement. That's because we have no code that uses those lines! Let's add them as a test. We need to execute the mapnik.VectorTile.addImageBuffer() function incorrectly, with the first parameter as something other than an object but has a length greater than 0, since you can see the if statement is checking for info.Length() < 1 || !info[0]->IsObject(). Using our test from above, let's write a new one:

it('throws an error when first parameter is not a buffer object in addImageBuffer', function(done) {
  // first create a vector tile object to work with
  var vtile = new mapnik.VectorTile(0,0,0);
  
  // assert.throws on addImageBuffer with the incorrect parameters
  assert.throws(function(){ vtile.addImageBuffer('not an object', 'name', function(err) {}); });
  
  // conclude the test
  done();
});

Remember that print statement we added to the C++ file? When we run the above test with make test we should see that logged to the console now, since we hit that particular set of code! Remember to remove the std::cout statement since that shouldn't make it past your local machine. Commit the test changes, and push up to the repository!

The it.only() trick

One thing that can make testing slow is constantly running make test and waiting for all of the tests to finish. Mocha has a fancy trick that allows you to run a single it test using the it.only() syntax. That looks like this:

// this test will run
it.only('throws error when X', function(done) {
  // tests
  done();
});

// this test will not run
it('does something else', function(done) {
  // tests
  done();
});