productive testing tips

I was cited in an article on testing tips this past week. Here is some cut and paste from the email I sent to the author (Joe Barr) when he asked me for tips; my email goes into more detail than he was able to put in his article so I thought it might be worth posting.

1. use a bugtracking system of some sort, and use as much metadata (including good titles) in that bugtracking system as you can. At first it is time-consuming, but over the long run, it’ll save you time by helping you avoid testing things or filing bugs twice. Relatedly, if we’re talking about things that can be done as a group, and not just as an individual, have a bugmaster- someone charged with organizing, sorting, and generally knowing what is going on with the bug tracker. That one person will help every other person who uses the bug tracker (both testers and developers) be more productive, which is invaluable.
2. Where possible, use the very latest code. Don’t be afraid to rebuild things from CVS every night or every morning. The moreup-to-date your code is, the quicker you’ll catch things (again, helping everyone’s productivity) and the less time you’ll spend going back and forth with ‘is this in the latest version?’ Again, when speaking of a team, if someone can be charged with making this process as easy as possible, that one person has a productivity multiplier for the entire team- makes everyone more productive.
3. If at all possible, write automated unit tests. The best way to be productive is to have the computer do the work for you. Again, this has up-front costs, but over the long run is a *huge* win. (If we’re talking about GUI software, try to focus on non-GUI code first when writing tests- GUI tests tend to be fragile, and you’re best off writing tests that will fail when things go badly wrong, not when a pixel shifted here or there.)
4. dogfood, dogfood, dogfood. If at all possible, use the code you’re testing under real-life conditions to do everyday work. Real-life testing is always going to be more efficient than fake ‘do this, then do this, then do this’ checklist testing- that doesn’t catch edge cases, and it feels like work. If you’re using code for real work, you catch the edge cases that real users find, and you do it in the courseof doing something else- it doesn’t feel like work then. (Relatedly: if you dogfood and find a bug, make sure to do (1) file it immediately, and (3) write an automated test to duplicate the bug ASAP.)
5. use automated crash reporting tools: every major OS now has ways to catch stack traces of catches and send them back to a bug database. Use those, and ship those- help the users help you.
6. if you have an active volunteer community who are using nightly builds to do daily work, don’t spend your time testing things that they will inevitably test. For example, I’ve seen test plans that say things like ‘check to make sure it launches’. If it doesn’t run from the command line, a good community will let you know about it ASAP. If it crashes when you open the print dialog with 10,000 printers on your local network, well, most communities don’t run into that sort of thing- they have one printer. So spend your precious test cycles testing *that* kind of scenario, instead of testing basic stuff our community will catch like ‘does it start? does the file open dialog work?’