We ran two meetings in appear.in - which worked fine for some of us, but not all. Will try Hangouts next time.
Present: Seb, Henry, James, Liam, Pawel, Gaspar, Alex
Henry:
- gave us an overview of how pitest got developed and some of the key performance challenges
- An extended write up can be found here
- There was a discussion about whether it would be best to mutate the IL or the AST.
- experiments/spikes seem to be required.
Pawel:
- was going to write his own mutation testing tool
James got cut off a couple of times, but what I heard him say was:
- the number of test frameworks we support would add to complexity
- there's nothing physical to port from pitest, but the learning & high level architecture could be used
- mutating the AST might exclude (or make more complex) supporting multiple CLR languages
- Roslyn supports direct manipulation of AST
- Ninja Turtle (now defunkt) took the IL approach
Liam:
- going with AST makes sense
- there may be nothing to gain from looking at pitest directly
- is there someone with deep CLR knowledge that could talk directly with Henry?
Alex:
- has produced a mature mutation testing tool for C/C++ and LLVM IL (?)
Gaspar:
- target .NET Core rather than classic .NET just incase there are significant portability issues
- C# is major CLR language, so don't let coverage of other languages unduly affect the IL/AST decision
Present: Seb, Simon, Peter, Stanislaw
Simon & Peter:
- Stryker.NET is in very early experimental stages
- currently can manipulate AST, but doesn't compile or run tests
- suggest targeting onlt MSTest initially
- have a domain
- would encourage people to help with the project
- will create issues/project items in Github to give new contributors a way in
- have some contacts at Microsoft who may be able/prepared to give us pertinent information
- will sketch out a high level architecture
- it would be good if the various mutation testing tools could integrate identically with common external frameworks (e.g. Jenkins SonarQube)
Stanislaw:
- working on C/C++ tool, so attending in 'read only' mode
- will share experiences
- seems there are a number of people who are interested in contributing
- Stryker.NET has already started (somewhat), so makes sense to contribute to that project
- there's a lot to be learnt from pitest
- the IL/AST question is important
- can the architecture make that decision [relatively] cheap to change, or do we need to get it right the first time
- we don't really have a handle on the runtime implications of the choices
- IL and AST spikes would seem to be an obvious first step, incorporating compilation
- Stryker is (effectively) a team of 2 work colleagues
- to absorb more contributors may be challenging