In a recent article I published at Software Test Professionals on Getting Started on the Right Foot for Quality Metrics, one comment I received was that it was regarding Table 1 indicating that different people are likely to be interested in different things with regard to software quality and test metrics, and that many stakeholders were not included in the table. What other stakeholders would you include?
Another comment regarding my list of “why metrics/measures fail” stating that since it was a listing of common reasons for failure, the implication is that there are other reasons.
Certainly there are many reasons for failure of metrics programs and the idea was brought up was dysfunction of the measure. Reliability and accuracy of measurements is definitely always a problem. And of course there are many reasons, just like a dysfunctional family. Everyone wants to ‘look good’, so when people try to make the measurement “look good, something else usually slips, perhaps not intentionally but because it is forgotten or overlooked while focusing what you are measuring. If you focus on productivity, then you may get lower quality. If you focus on defect counts, then that’s what you’ll get, lots of defects. If you focus on software performance, then your software may get faster on some functions but slower in others. In the end, this may cause people distrust and ignore all metrics/measures once they see the altered and non-beneficial results. This is sad because metrics are a great way to manage and measure progress and get our job done. Just think of your own health. Can you imagine where you’d be without a scale, or routine blood test to check your cholesterol? But just like those measurements, they have to be done several times and consistently to make sense. You can’t eat breakfast one day and get a blood test and then the next time, fast overnight… The same applies to calculations and measurements taken for software quality and test metrics. You need a framework and consistency. Make sure you fast the night before to get accurate readings.