In the recent webinar “Software Quality Metrics – Do’s and Don’ts“, with ASTQB, we ran a few polls to get an idea of how software quality professionals are really using, OR NOT Utilizing Software Quality Metrics. It turns out that:

Most of those surveyed collect 1-5 software quality metrics

Those collecting 1-5 metrics were the majority.

Most people are collecting some metrics, with the majority collecting 1-5 metrics. That’s a great start, and depending on the resources available, we recommend 6-10 with 1-2 metrics collected on each process component.




Collecting software unused software quality metrics

Many collect software quality metrics that are not used.

Almost 2/3 of those surveyed collected metrics but then ended up not even using them!




Metrics Too Difficult To Collect

How many of you started collecting metrics and then found it was too time consuming and quit?

Many found that utilizing software quality metrics, while ‘nice’ and good information, is just TOO HARD. Many started and quit.




The webinar was quite lively with lots of people posting questions but not enough time to answer them…Questions included (which will be answered in future posts) :

Any suggestions for measuring resource allocation?
Can you also please share some thoughts on gathering soft information such as lost business opportunity, etc.
Do you have any recommendations for leading indicators of quality during software development.   Most of the ones on the slides appeared to be lag indicators.
How do you balance (or minimize) the effort involved in collecting and summarizing the results of the metrics to impact to the project work being done?
How do you determine system size?
It’s hard to calculate cost for test activities.
What is the general improvement (cost %, time % of overall budget/duration of project) seen by implementing a measurements & metrics framework across the industry? This will help make the business case for implementing measurement framework, can you expand on the investment to benefit ratio of implementing a metrics framework?
Do you measure project estimates? If so, how?
Do you plan to cover balanced score card in this webinar?
We are collecting metrics not used because the organization is not mature enough yet.  We are doing this to drive things forward and show the maturity of QA group.
What did the acronym KLOC/FP mean listed in the slides.
How important is defect density as a measurement?
What recommendations do you suggest that we include on a weekly QA scorecard?
will you discuss how to generate metrics per function point?
What measurements do you use to measure project delivery?