During the ASTQB 3 hour workshop, Software Testing Metrics Are For Action, one of the top questions was the use of DRE, or Defect Removal Efficiency. I was surprised to find that out of the 40 participants only 5 used this metric (12%). This number was similar to a workshop I was in last year at PNSQC where the material presented indicated that even less than 12% considered this metric essential in measuring as part of a metrics program. The class was lively and spirited. We divided the class into 3 modules:

Defining software quality: With so many different ideas on what quality is, we first ventured into understanding the different viewpoints of what software quality is and how organizations use this definition, OR don’t use! We covered many different software quality models to help people get their hands around and understand what software quality is.

Measuring with metrics: Because I didn’t know the level of all these students, I started with basic concepts like the difference between a measurement, metric and indicator. This may have been boring for some, but it turned out many did not know the difference and they seemed quite happy to discover that they were only collecting measurements and not metrics. We also covered interpreting metrics and how this must be done with knowledge and understanding of the context. We also covered many of the software quality and software testing metrics, discussing each one in depth.

Metrics frameworks: Lastly, we pulled all the metrics into a framework, showing the participants how to create and customize their own framework. Frameworks are needed to make sure we are measuring the right things for the right people. One of the more well known, GQM, Goal-Question-Metric, was covered, but we introduced the students to several others.

We covered lots of material. At least 2 of the attendees said that it could be a full day tutorial (This was a half day session at the ASTQB Testing Conference). I’ll be doing it again for a full day at the Quest Conference in Baltimore in 2 weeks. I’m looking forward to discussing defect removal efficiency again and taking the poll!