December 13, 2007

On Automated Testing

Test everything somewhere. Test nothing everywhere.
                             -- Andrew Fuqua, December 2007

We were talking about automated testing, particularly test coverage, in Agile Atlanta the other day when I coined that aphorism. At least I don't think I've heard it anywhere else before. The concept had been in my head forever and I've worded it similarly thousands of times, but perhaps not so succinctly.

When writing test code it's good to have almost everything tested in some fashion. But there is little reason to test anything multiple times. Just cover everything well, in one layer or another. Any concept well covered in unit tests doesn't need to be tested in higher layer functional tests. Any concept well covered in integration tests doesn't need to be tested in customer acceptance tests. It would be inefficient to cover the same area so many times.

Of course, it's okay to leave certain cases to be tested by hand, as long as they are relatively few in number. Leave what can't be automated easily for manual testing.