Fixing Test Track Tennis was done by agreeing a few simple rules with both the development and testing teams. The rules, which I’ll spell out shortly, were only part of the solution though. A much bigger contributor to the death of test-track tennis arose more as an intended side-effect of the rules: in following the rules, the developers and testers spent a lot more time talking to each other about both the testing and development processes, and over a small number of months the atmosphere changed from one of antagonism between test and development to one of collaboration.
The rules were:
- There’s no such thing as a “No Repro”.
- Once you’ve fixed your bit of code, test to the end of the script.
We removed “No Repro” as a defect resolution option in the system. Any developer that couldn’t quickly reproduce a defect assigned to him was to immediately get off his backside, walk to the tester’s desk and ask the tester to demonstrate the repro case. Occasionally, a tester would be less than spectacular at describing repro cases, so occasionally defects would get referred to me (as head of development) so that I could have a conversation with the head of testing and the relevant tester could get some coaching.
Defects that only occurred intermittently would be flagged as “intermittent” on the system, and both development and testing teams would keep an eye out for other factors that made the defect more or less likely to occur. The defect would sit with the developer until a solid repro case was found, and checked every time nearby code was edited until things became more clear.
Testers would include the details of their current test script in the defect record, so that the fixing developer could test it right to the end. If he found another bug in his own code, obviously he’d fix that too. However, if he found a bug in someone else’s code, he’d give them the defect to fix, and they’d give it back to him to complete the test. The developer that fixed the first bug in the defect OWNED that defect, until the test script ran right to the end without errors.
This scheme took the average time-to-fix for defects down from six releases to one release. Additionally, it reduced the incidence of “Not Fixed” defects from about 35% down to less than 5%. Maybe not as low as we’d have liked, but low enough that I could take a personal interest in each one that came back like that, which then fed into the coaching and personal development delivered to our developers.
Quality went up. Morale went up. Productivity went up.
This was my first “win” as a newbie development manager, which is probably why it’s stuck with me for so long. Feel free to comment on your first win (or loss), or on ways to improve this strategy the next time round.