Have you implemented a software quality metrics program? If so, then you know it’s not easy and that metrics programs often fail. Why?
Software Quality Metrics – What Am I Possibly Doing Wrong?
- Metrics don’t measure the right thing – Often times, we get an idea for a software quality metric from a person, company or article and begin using it without thinking ‘What am I trying to measure and why?” In the end, we sometimes get measurements that don’t matter relative to our goal.
- Metric becomes the goal – It is possible for organizations to be so concentrated on the metrics, that they don’t understand the metric’s relationship to the goal or objective. If you are measuring software quality, certainly defect counts matter. But there is a lot more to it, and defect counts need to be incorporated into an overall valuation.
- Metrics don’t have consistent context so they are unreliable – Just as in weighing yourself, it doesn’t make sense to drink 2 gallons one day and weigh in, and go jogging 10 miles the next day and weigh in. Context needs to be defined and then maintained for measurements to be meaningful. This can be difficult in today’s environment with changing test platforms and other contextual factors.
- Measurements used are not consistent – Just as context needs to be consistent, so do the measurements and the methods that you collect the measurements and calculate the metrics.
- Metrics don’t answer the questions you had to begin with – You run off collecting measurements and calculating metrics without thinking what answers will I get after getting this information?
- Metrics have no audience – As a corollary to the previous factor, if there is no question to be answered, then there will also be no audience for the metric. Metrics need to have an audience in order to have meaning. How many of the metrics and reports that you generate are read?
- Measurements are too hard to get – If you end up designing the right metric to answer the right question, it doesn’t matter if it takes several man days to get the data and do the calculations. Unless the value and decisions made from these metrics have considerable value, they’ll soon be abandoned.
- Metrics have no indicators so cannot evaluate – You collect mounds of data but then what? How do you determine what is ‘good’ or ‘bad’? Before you get started collecting and calculating you need to put together a way to evaluate the numbers you get with meaningful indicators that can be used as benchmarks as your metrics program matures.
As you can see, designing and implementing a software quality metrics program requires careful thought and planning. The first step is finding out the questions that the software quality metrics will answer and to differentiate them from software testing metrics. Many refer to this as the goal-question-metric paradigm, but in simple terms, what are you going to do with the numbers once you get them? Fill in the form below to find out more about how XBOSoft can help you design and implement a long running metrics program.