Rough draft - 2015-08-25
- Burndown rate (not velocity)
- Rejection rate
- Defect (bug) rate
- Ratio of lines of test:lines of code >= 2:1
- Volatility
Burndown rate is measured by tallying the amount of outstanding work required for a release at the end of each sprint. This is different from velocity that only measures the volume of work completed per sprint. It's possible that velocity can show that work is being completed in the sprint, but with more work continually being added to the scope of a release the burndown can show that more work is actually outstanding at the end of each sprint than at the beginning.
If work items are being completed by the engineering teams but regularly get rejected at the point of acceptance testing then there's a lot of waste happening. Either there's a breakdown in communication between customers and engineers since the engineers don't understand the customer's requirements, or the engineering team's goals aren't aligned with the overall team's goals.
If work passes acceptance testing but is later found to be defective it causes even more waste than just rejection, since time would have passed since the original work was done and a different person could be assigned to work on the defect. The extra time taken for context switching would have been better spent by being more meticulous about testing the work initially. Defects must cut into the overall burndown rate since the time allocated in the original estimation has already been accounted for.
As software grows, there are bound to be several paths through the code. Healthy projects tend to have a ratio of at least 2 lines of tests for each line of code. This is not a hard-and-fast rule that has to be religiously held to, but if there is a low ratio of testing code then there must be a good reason for it. If there's a high number of integration / system acceptance tests and a low ratio of unit tests then it's a flag that the internal code quality is low, even if the external quality is high.
The variance of burndown-rate from sprint to sprint. Volatility measures the probability that you will meet the same burndown rate for the current sprint. This normally gets measured over 3-5 sprints by measuring difference between the actual burndown rate and the projected burndown rate based on the past sprints. If you're tracking volatility over 3 sprints and you've hit exactly 20 points for the past 3 sprints, your volatility is 0%. If you only hit 18 points in one of the sprints then the volatility is 3.3% (1 - (58/60)). Likewise if you hit 22 points in one of the sprints then the volatility is still 3.3% (1 - (62/60)).