Metric Rhetoric

I meet very few testers who argue that there is value in metrics like number of test cases executed as a measure of progress. The rhetoric for metrics such as these seems to have changed from defiant oblivion to a disillusioned acceptance; "I know it's wrong, but I don't want to fight".

The pervasive view is that, no matter what, project management will insist upon metrics in test reporting. Testers look on in awe at reporting in other areas of the project, where conversations convey valuable information. We are right to feel sad when the reporting sought is a numeric summary.

But they want numbers...

Do they? Do your project management really want numbers? I hear often that testers have to report metrics because that is what is demanded by management; that this is how things are in the real world. I don't think this is true at all. Managers want the numbers because that's the only way they know to ask "How is testing going?". If you were to answer that question, as frequently as possible, making testing a transparent activity, much of the demand for numbers would disappear.

"Even if I convince my manager, they still need the numbers for reporting further up the chain". If your manager has a clear picture of where testing is at, then they can summarise this for an executive audience without resorting to numbers. Managers are capable communicators.

Meh. It's not doing any harm.

Everytime that you give a manager a number that you know could be misleading or misinterpreted you are part of the problem. You may argue that the manager never reads the numbers, that they always ask "How's it going?" as you hand them the report, that they'll copy and paste those digits to another set of people who don't really read the numbers either, so where's the harm in that? Your actions are the reason that I am constantly re-educating people about what test reporting looks like.

Snap out of it!

We need to move past disillusioned acceptance to the final stage and become determined activists. A tester is hired to deliver a service as a professional. We should retain ownership of how our service is reported. If your test reporting troubles your conscience, or if you don't believe those metrics are telling an accurate story, then change what you are doing. When asked to provide numbers, offer an alternative. You may be surprised at how little resistance you encounter.