-
TestNG - test framework for Java
-
- test framework for R
-
Use mocks/stubs for isolating parts of your code in testing
- Lots of mocking frameworks available (depends on your choice of language)
-
ioprofiler - parses and displays lists of system calls recorded by strace
-
Selenium - browser automation framework for integration/acceptance testing web apps
-
gcov - test coverage tool integrated with GCC (i.e. make sure you're testing every code path)
- there are similar tools for other languages
-
Test-driven development (TDD) - more of a philosophy of describing expected behaviour as tests first, then writing the implementation code after
-
Vagrant - useful for isolating environments to figure out why tests are sometimes failing and sometimes passing
-
bisect - command in various version control systems (git, hg at least) that helps identifying the commit where a bug was introduced
-
Jenkins - service that automatically runs your tests (continuous integration) in various environments and tells you (e.g. by email) what's failed
- Also look at TravisCI if your needs are quite simple
-
Trac - bug tracking/ticketing software
- Other bug trackers are available: github has bug tracking and code-review features; even a board full of post-it notes can help
-
Testing is regarded as tricky with research, because you don't necessarily know what the right output should be yet
- Unit testing can help, because you should know what each individual part is meant to do
- Test driven development (TDD) can be useful too, because it gives you a structure to think about what problem you're solving in a different way
-
Process of becoming a developer in research is often quite organic
- Doing things in silico becomes interesting from a research perspective, so you teach yourself to code
- As long as the results look reasonable, no-one questions the quality of the software
-
Precedents exist for how data/results are presented and published, but no such precedent exists for code
-
Software QA needs to become part of peer review
-
At publication
- Are results scientifically valid without software QA
-
At grant application stage
- How will you ensure your software does what it's meant to?
- There isn't always acceptance by RCs reviewing project plans
that softare QA is worth spending time on
- I.e. they will see a bid including QA as being too long/expensive
-
-
Testing is very rigorous and built into the process in industry
- You cannot bid for work without a software development plan and quality plan
-
Tests are written before any application logic
-
Regression testing (does new code produce the same results as old code) seems most common
-
Unit testing less common
- Sometimes someone starts, but testing falls by the wayside
-
Retrospectively adding tests is really hard!
- Introduce TDD to beginner programmers at the very beginning
- Software Carpentry already does this
-
Testing allows refactoring with confidence
- Removes the problem of old code that people daren't touch
-
Would be good to share stories and evidence that testing is worth it
-
Weekly "come and share battle stories" sessions can work well
- These can end up turning into standard seminar sessions, but can still be valuable
-
Write one regression test
- Does the code still do today what it did yesterday?
-
Write some high-level designs
- Start with actually writing down what you want to achieve
-
Set up continuous integration
- Simplest case - does the code actually build?
- Massive time required to add testing to large extant codebases
- Ways to start:
- When you write new code, write tests for that first
- When you find a bug, write a test for that first
- Demand that bugs are not fixed without a test
- How do you QA third-party code to run on your platform?
- Where do you start with an existing culture of not testing and a legacy codebase?