[ZZ]google v. microsoft, and the dev:test ratio debate
Every since I gave a talk at Google’s GTAC event here in Seattle this past October, I’ve had the chance to interact with a number of Google testers comparing and contrasting our two companies’ approach to testing. It’s been a good exchange.
Now it seems that, their toilets notwithstanding, Google focuses on testing with an intensity that is in the same general ballpark as ours. We both take the discipline and the people who do it seriously. But I think that there are some insights into the differences that are worth pondering.
Specifically, the disparity between our respective developer-to-tester ratios is worth a deeper look. At Microsoft the dev:test ratio varies somewhat from near 1:1 in some groups to double or triple that in others. At Google just the opposite seems to be the case with a single tester responsible for a larger number of bug-writing devs (clearly we have that in common).
So which is better? You tell me, but here are my thoughts (without admission of any guilt on Microsoft’s part or accusations against Google):
1. 1:1 is good. It shows the importance we place on the test profession and frees developers to think about development tasks and getting the in-the-small programming right. It maximizes the number of people on a project actively thinking about quality. It speeds feature development because much of the last minute perfecting of a program can be done by testers. And it emphasizes tester independence, minimizing the bias that keeps developers from effectively testing their own code.
2. 1:1 is bad. It’s an excuse for developers to drop all thoughts of quality because that is someone else’s job. Devs can just build the mainline functionality and leave the error checking and boring parts to the testers.
It’s interesting to note that Microsoft testers tend to be very savvy developers and are often just as capable of fixing bugs as they are of finding bugs. But when they do so, do devs really learn from their mistakes when they have someone else cleaning up after them? Are testers, when talented and plentiful, an excuse for devs to be lazy? That’s the other side of this debate:
3. Many:1 is good. When testers are scarce, it forces developers to take a more active role in quality and increases the testability and initial quality of the code they write. We can have fewer testers because we our need is less.
4. Many:1 is bad. It stretches testers too thin. Developers are creators by nature and you need a certain number of people to take the negative viewpoint or you’re going to miss things. Testing is simply too complicated for such a small number of testers. Developers approach testing with the wrong, creationist attitude and are doomed to be ineffective.
So where’s the sweet spot? Clearly there are application specific influences in that big server apps require more specialized and numerous testers. But is there some general way to get the mix of testers, developers, unit testing, automated testing and manual testing right? I think it is important that we start paying attention to how much work there really is in quality assurance and what roles are most impactful and where. Test managers should be trying to find that sweet spot.