Category: Technique
Level: Intermediate
Session Type: Talk
Automated testing is key to ensuring the ongoing health and wellbeing of any software project, giving developers and users confidence that their software works as intended. Property based testing is a significant step forward compared to traditional unit tests, exercising code with randomly generated inputs and ensuring that key properties hold. However, both of these techniques tend to be used at the level of individual functions. Many important properties of an application only appear at a higher level, and depend on the state of the application under test. The Haskell library hedgehog, a relative newcomer to the property based testing world, includes facilities for property-based state machine testing, giving developers a foundation on which to build these more complicated tests.
In this talk, Andrew will give you an introduction to state machine property testing using hedgehog. He'll start with a quick refresher on property based testing, followed by a brief introduction to state machines and the sorts of applications they can be used to model. From there, he'll take you on a guided tour of hedgehog's state machine testing facilities. Finally, Andrew will present a series of examples to show off what you can do and hopefully give you enough ideas to start applying this tool to your own projects. The application being tested will be a servant web application, and examples will include testing fundamentals such as content creation and deletion, uniqueness constraints, and authentication.
An intermediate knowledge of Haskell is expected, and familiarity with property based testing will be beneficial. The slides and demo application will be available after the talk for people to study in detail.
The target audience is developers working on applications that can be modelled using state machines (e.g. web applications), and who want to test more complex properties of their software.
A basic understanding of property based testing, ideally using the hedgehog library, is expected; as is a basic understanding of web programming. Knowledge of Haskell is beneficial, but not necessary.
The basic outline of the talk is as follows
- Introduce myself and work context in which this has been used.
- Quick refresher of/intro to hedgehog and property based testing. As this is expected knowledge, this will be very brief.
- An introduction to state machine property testing in hedgehog.
- What is a state machine?
- Examples of apps that can be modeled as state machines.
- Overview of hedgehog's approach and key types/functions.
- Example 1: a toy example of an in-memory system with two states.
- Example 2: a CRUD web application.
It is intended that example 2 builds in difficulty. I do not have an exact list of properties, as this will depend on timing and feedback from test runs. Below is a list of some properties and their progression to give a feel for what I have in mind. I doubt I'll have time to cover everything in this list, however I hope it demonstrates that there are plenty of examples to draw on.
- Creating an item increases the number of items when listed.
- Deleting an item decreases the count of items when listed.
- Inserted items are retrieved unchanged.
- Deleted items cannot be retrieved.
- Inserting data that would violate a uniqueness constraint does not insert the data and returns the correct error.
- Unauthenticated users can't access protected content and receive the correct error.
- Authenticated users can access protected content.
- Users can't login to the same account multiple times. . Example 1 might be skipped in the interests of time, in which case the first property demonstrated in the web app example will be very simple.
- Introduction to the idea of state machine property testing
- Identifying applications that can be modelled as state machines
- Identifying and specifying properties of state machines
- How to use the hedgehog package to write state machine property tests, including tests for web applications.
haskell, property based testing, state machine testing, hedgehog, servant