Yesterday, I finished my presentation at Starwest on Software Quality Metrics For Testers. It was a packed room and I was really surprised. I started by discussing the title, why ‘quality metrics’ instead of ‘testing metrics’. I like to use analogies, so I started out by discussing diet and health, something everyone can connect with. With diet it’s easy to see that what we eat affects our weight, and our weight in turn impacts our performance and results if we engage in sports. Using that we can compare that to testing, where activities done well early in the development lifecycle such as requirements can influence code quality, and satisfaction of requirements. This in turn, can impact the test results, and in the end, the users.

Software Quality at different phases

One stage of quality affects the next stage.

What I stressed most in the talk probably is that many metrics programs fail because they don’t answer the questions of stakeholders. If we you don’t answer these questions via the metrics we create, then what are we reporting these numbers for? For fun I guess?

The second take away from the presentation was how to develop a framework to hold your metrics so that they are organized in a way that matches your process and organizational structure. Without a framework, it can be hard to see what we are missing, or that perhaps all our metrics are only counting one thing… Defects usually.

Software Quality Metrics Framework

Software Quality Metrics Framework

The final take away that I discussed was the concept of libraries or catalogs that you can develop and accumulate over time. These catalogs can include:

  • stakeholder questions from all stakeholders since the CEO is interested in different things than a development manager.
  • stakeholder objectives to use as buckets for organizational purposes
  • answers that address the questions
  • metrics that address those answers

With such a crowded room (standing room only), I’ll definitely refine my talk and have some flexible modules that can be pulled in depending on the audience. I’ll be giving many talks and workshops on software quality metrics in the near future include QUEST 2014 in Baltimore and ASTQB in San Francisco, and others. Stay tuned.