Title: The plural of anecdote is not test suite
Type: About half a talk, with the rest of the time for live demos (assuming the demo gods are kind, if not I'll have some bonus slides to talk through) and questions. Experience suggests that talking on this subject gets a lot of questions.
Duration: 90 minutes
Speaker name: David R. MacIver
David is the primary author of Hypothesis, a property-based testing tool for Python whose stated mission is "to drag the world kicking and screaming into a new and terrifying age of high quality software". It's a work in progress. He's also working on Conjecture, whose stated mission is "Port Hypothesis to every programming language". Prior to tilting at windmills, he worked at Google and a series of London startups as a back end data engineer.
Location: Cambridge, England
A test suite should constrain the behaviour of your program to be as close to correct as possible. The best test suites are a kind of executable specification of your code's behaviour.
Most test suites are instead a series of anecdotes about this thing that happened that one time.
We try to fix this problem by adding more anecdotes, because we don't have the tools to turn our test suites into full specifications.
In property-based testing we work in partnership with the computer: We write the tests but the computer supplies the examples to check them. So instead of writing about that thing that happened we get to specify what should happen for every example we've been provided.
So why don't we do this? Property-based testing tools have existed for years, in the form of property-based testing libraries like QuickCheck. Instead of testing like it's 1989, why don't we test like it's the year 2000? Or maybe even later?
Unfortunately, the tools to do so have been stuck in functional programming land - if you're writing Haskell, Erlang, Scala, Clojure or one of a number of others, your life is great. If you're like the rest of us, not so much. This is what Hypothesis and Conjecture aim to fix.
In this session we'll explore property-based testing in mainstream languages - mostly Python through the medium of Hypothesis, but we'll also look at it in other languages. This will be a mix of high level talk about the whys, some detailed walkthroughs of examples, and a whole bunch of live demos where you can put me on the spot and ask me to test things.
Expected level of participants/audience: Beginner friendly, but intermediate and above will probably get more out of it. Some Python familiarity would be helpful, but there will be examples in a variety of languages (TBD, but most likely Python, Java and C++).