I’m the project lead on Lisky, the command-line interface we’re building for developers to interact with Lisk nodes and perform other Lisk-related functions via the command line. We’ve recently adopted a BDD-style approach to unit tests in Lisky (from v0.3.0 onwards), and this blog post is for anyone interested in how that works, in particular anyone interested in contributing to the Lisky codebase - pull requests welcome!
What we’ll cover in this post:
- What is BDD?
- A tutorial-style step-by-step guide to writing your own tests using this approach.
- What are the benefits and what are the disadvantages?
BDD stands for behaviour-driven development. When it comes to defining approaches to automated testing, sources can vary widely, but as understood here BDD incorporates ideas from domain-driven design (DDD) into a test-driven development (TDD) process.
So much has been written about these concepts elsewhere, so I won’t go into too much depth here. If any of these terms are new to you, don’t worry about it—the easiest way to understand how the approach works is just to work through the examples in the tutorial below.
In brief though: TDD is an approach to software development in which the developer first writes tests which define the desired behaviour, and then writes code which passes the tests. DDD is an approach which emphasises a consistent, implementation-neutral domain language. Thus the form of BDD we’ve adopted involves these three steps:
- Writing an executable specification consisting of a series of steps described in implementation-neutral domain language
- Writing test code which implements each such step atomically
- Writing source code to pass the tests (and thus conform to the specification)
As with the Gherkin language most often used for end-to-end testing, we divide specifications into
- Given (for setting up test context),
- When (for execution of the code under test), and
- Then (for making assertions) steps.
OK, let’s try writing tests for a function that takes a name and a language and wishes that person happy birthday in the selected language. There’s a companion repo with all the code described in this blogpost in case you get lost at any point. The commit history matches the progression outlined here, so you can step back to exactly the point you need.
I’ll assume you have Node and NPM installed and are comfortable using the command line. I’m using Node v8.9.0 and NPM v5.5.1, so if you run into difficulties below check if using those exact versions helps. The code below is written in ES6, so I’m assuming you’re comfortable with that already.
Create a directory and navigate inside it:
https://gist.github.com/a9cbfc3ebcb2a0c9897d41824b1c03cc
You’ll need Mocha installed to run the tests:
https://gist.github.com/d0b77090b1b62c115d456e6c828e6875
No need to use the --save or --save-dev options, once installed we can run Mocha directly using npx.
Finally we need some files to store our code:
https://gist.github.com/7ea9e04f89af3eaf7ec18f766e1b28d2
In the code blocks below I’ll put the name of the file being edited in a comment at the top, and indicate when I’m eliding code with a // ... comment.
We’ll use an outside-in approach approach considering the happy path first. "Outside-in" means we’re going to write the code we really want to work first, and write the code it needs to work later and only when we’re forced to. This contrasts with "inside-out" which involves trying to predict the code you’ll need later on, so that when you come to write that code you already have all the code it depends on. Of course, you may find an inside-out approach suits you better, but outside-in works especially well with BDD.
Addressing the happy path, we start by specifying what should happen if everything goes according to plan:
https://gist.github.com/421a2edb2826aa8f3fac03ad367806eb
Now running Mocha on the specification should show us a pending test because then.itShouldReturn is undefined:
https://gist.github.com/5006f9894478dfdfcc326c87625ce968
We need a step definition!
https://gist.github.com/397a728ddc02bf855152d9a46d2d835d
What’s happening here?
- We’re exporting a function from
then.jsfor the specification to refer to. - That function asserts that the return value is equal to some expected value (simply doing what the function name implies).
- The return value is destructured out of the test context (this is just a feature of Mocha).
- The expected value is extracted out of the test title using a regular expression.
The last point here allows us to introduce specific examples of certain values in our specification, which are then used directly in our tests, so it’s easy to see whether a specification is working with realistic values, and we don’t have to worry about keeping redundant definitions synchronised with each other. It also means this step is ready for reuse in a different test with a different string value.
Now we have a failing test:
https://gist.github.com/31040b82b10d63b8cc0645f9b75fe627
Of course returnValue is undefined, because we haven’t defined it yet. We need to develop our specification:
https://gist.github.com/5cb8f3ca8c71936e79e110a31e68ff74
And the corresponding step definition:
https://gist.github.com/ae8dfdd9180465f7bd3486f36693b8ef
Here we store the return value in the test context so the Then step can access it later. The test fails: TypeError: wishHappyBirthday is not a function. name and language will obviously have to be dealt with at some point, but right now we need a source code function!
https://gist.github.com/a3132d28638610479d848c0ad372a4b5
Our tests are passing:
https://gist.github.com/3a61d076202bc5851317c3de2dbd3f42
But this function is terrible, it gives the same output regardless of name or language. We need a richer specification:
https://gist.github.com/64e348b1e10b7ef68d658f2c61f4db26
And corresponding step definitions:
https://gist.github.com/ed01ea06b4d20059b09527ca90c4e7c9
Here we reuse the getFirstQuotedString function we saw earlier to get the name/language from the test title and store it in the test context for later access. (You could store that function in a utils file if you wanted.) Note that because these functions are run in a beforeEach hook, we have to use the title from the test parent, not the test itself.
We’ve nested parts of the specification to cover both names in both languages (2x2 scenarios). Notice also that even though we added six totally new steps to the specification with a bunch of different variables, we only had to write two step definition functions. Now that’s what I call DRY!
We get three failures along the following lines:
https://gist.github.com/cb3ab73b6cc5fa49469d095a4ce674a3
We have no choice but to improve our source code:
https://gist.github.com/b485f88b7787fc963665130aa02c5eca
And our tests pass!
https://gist.github.com/3920a6cfd551808ea98ffd418052a43a
So much for the happy path, what about when things go wrong, such as if someone calls the function using a language we haven’t handled yet? Let’s update the specification first:
https://gist.github.com/571508273c42e4ef73ffc640a497b4f9
Then the step definitions:
https://gist.github.com/98cb58286989dcd863e83f5c07f39809
https://gist.github.com/fe06f0419f699020feb2933c2d5b56d5
The test doesn’t care if the language is known or not, so we can just alias given.anUnknownLanguage to the given.aLanguage step definition we already wrote. We do need to update our when.wishHappyBirthdayIsCalledWithTheNameAndTheLanguage step definition though, so that if an error is thrown it’s stored in the test context for later access:
https://gist.github.com/b01786941b6bf5d599d05d3095a2a78a
The test fails with TypeError: Cannot read property 'message' of undefined because our function doesn’t throw an error at all, let alone one with the right message. Time to update the source code:
https://gist.github.com/a6253b6d262de641b61e92c8989204d2
And everything is passing:
https://gist.github.com/8a5f6b376f6ff1bafc517dd08e50db55
Obviously there’s a lot more you could do in terms of validation for this function (missing names, names with the wrong type etc), but we’ll leave this tutorial here.
- This approach enforces a consistent structure/style for your tests. Specifications have a reliable look and feel, and step definition functions end up being short and self-contained. No more spaghetti tests!
- It results in atomic tests by default, so your test suite is less brittle.
- Writing specifications with language abstracted from test implementation allows you to think about the exact functionality you want without getting distracted by thoughts about how you will test that functionality. This encourages stronger, more meaningful tests, which should ultimately result in more robust source code.
- Once you’ve properly thought about what steps are required for some test, it’s usually trivial to write the actual test code.
- It’s also easier for newcomers to the codebase to understand the tests you’ve written: instead of being forced to infer the meaning of a test from the implementation, a new developer simply reads English-like sentences which explain what’s happening in easily digestible chunks. They’re helped by the fact that test descriptions are verbose and explicit: the higher degree of repetition in specification files is a cost paid for the benefit of intelligibility.
- Since each step is defined in one place, this approach encourages reuse of existing steps, resulting in a more DRY codebase. We removed (net) hundreds of lines of test code when switching from our old testing approach in Lisky.
- Refactoring often poses problems for tests: they can break in a way that requires a lot of updates all over the place or in the most insidious cases they can lose meaning without you noticing. This approach makes refactoring test code much simpler - just make the change to the relevant step definition and all of the tests which include that step reflect the update.
- Writing good specifications is hard. Actually this is true of other testing approaches too, but it’s perhaps more immediately obvious when you’re doing BDD.
- Sometimes it can be difficult to tell when you’ve already defined a step that you’re using in a new test. This can be mitigated by splitting your step definitions into modules so you can easily browse through a shortlist of potentially applicable step definitions before writing a new one.
- It’s a little unconventional, so developers new to the approach may take a while getting used to it.
This BDD approach to tests is relatively new to us, and we’re still getting used to working with it. As we get more comfortable we’re discovering more patterns, codebase management techniques, and better ways to approach creating steps. But we’ve already seen benefits in terms of clearer tests and a more concise test codebase.
I take it as a good sign that when it comes to writing tests in other projects which haven’t adopted this approach, it now feels frustratingly unstructured, almost as if it’s inviting you to write lazy tests.
If you’re interested in contributing to Lisky, we’d love to hear from you! We have some contribution guidelines, and we ask pull requests to include full test coverage (using the BDD style described in this blog post). There are some divergences from the code used in this tutorial to note though:
Happy testing!
- Konstantin Kudryashov on Modelling by Example (and a video)
- Testing README in the Lisky project