Defect Removal Effectiveness – Agile Testing Webinar Q&A

In our recent agile testing webinar, we had an outstanding question on Defect Removal Effectiveness (DRE).

Query1:

For DRE calculation, I believe below is the correct measure. Can you please double confirm?

DRE = Defects found pre release/DFP+Defects found in prerelease

Yes, your calculation is correct. Ours is actually the inverse and would be called Defect Escape Rate (wanting to keep low) instead of Defect Removal Effectiveness.

Let’s do an example just to illustrate.

Let’s suppose you found 120 defects pre release.

Then after you released the software, let’s say, 15 defects in one month.

Therefore, the calculation is:

120/(120+15) = 88.88

This would put you in the middle of the pack.

Query2:

Since we are talking about agile, do we really need to specify 90 days of period here?

This is an excellent question. With agile, although you may work in 2-week sprints, and demonstrate ‘working software’ to your Product Owner, that doesn’t mean you necessarily ‘release’ to the end users. So, if you are doing a real release every 2 weeks, then the 90 days would not make sense. With most of our clients, they may not decide to actually release the software in sync with their sprints, but something slower. In any case, you need to be consistent. Defects found in production during a 1 week period is obviously going to be smaller that found in a 90 day period. Also, you have to think about what kind of defects you count, i.e. P1-P4, and whether you weight them all evenly.

To get more info, on agile testing and metrics, you may want to download our agile test plan white paper which has contains items you should include in an agile test plan and discusses what you don’t need too!

2 Comments

  1. Meeakshi Agarwal April 4, 2018 at 8:09 am - Reply

    We’ve recently moved to have 2-week sprints and observing good results in terms of productivity. But what’s your opinion on system testing? What’s the right agile strategy to perform it?

    • Philip Lew April 6, 2018 at 3:35 pm - Reply

      Each sprint usually encompasses the testing for those features and functions within the sprint. However, you are right. Where does the entire system and integration testing go? You cannot fit all that into a sprint. Several different ways to approach this. Some organizations reserve an entire sprint for system integration testing, or you could have a separate group of a few people who are constantly doing test automation and system integration testing. Otherwise, you will basically build up technical debt.

Leave a Reply

Show Buttons
Hide Buttons