Nowadays, many companies consider integrating TPI into their testing processes. However the questions of how much does a TPI effort cost and will these costs exceed its benefits have been raised by many practitioners. This article discusses and elaborates TPI’s benefits and costs, so that you can decide whether TPI is justified.

Most of the time, a TPI effort is triggered by one for more of the following three testing issues:

  1. Testing takes too long
  2. Testing is too expensive
  3. False expectation between developers and testers

The primary goal for TPI is to improve the product’s quality through making the testing process more efficient and effective. However besides achieving these goals, management needs to be aware that the costs of TPI sometimes are greater than its perceived and immediate benefits. Consequently, management should choose to implement TPI only when it satisfies the Cost & Benefit Formula presented below:

TPI Benefits – TPI Costs >> “do nothing” Benefits – “do nothing” Costs (problems)

As a result, it is critical to properly measure and balance the costs and benefits, especially the long term benefits that can be achieved through TPI and the potential costs from “do nothing”. Some common TPI benefits and costs are summarized in the table below:

TPI Tangible Benefits TPI Tangible Costs
Better software qualityAssessment aspects are time consuming
More control on the test process, leading to shorter test cyclesDevelopment time for new methods, procedures and templates
Increase in return on investmentIntroduction of test tools
Improvement of personnelTraining and pilot projects
Better efficiency across the software development lifecycleMonitoring and reporting on improvements
Enhance the software’s reliabilityFunds spent on consultation and changes
TPI Intangible Benefits“Do Nothing” Intangible Costs
Raise market awareness & reputationReduce company’s reputation
Improve quality & customer satisfactionLose current and potential customers

Companies should use pre-defined measurements and indicators to evaluate these costs and benefits in a tangible and quantifiable way. This can be difficult as these measurements usually depend on making assumptions combined with doing a sensitivity analysis. Balancing the analysis’ complexity so that it is not too difficult to understand yet can still derive believable estimates can be tricky. I’ll be discussing how to do this, step by step, in a future blog post.