As mentioned in a previous post on End User Perspective of Quality, quality means much more than conformance to requirements and the end user perspective on quality, sometimes called the User Experience (UX), is becoming more and more important. Measuring UX is a tough task but it’s important any attempt to quantify end-user quality and quality in general.

Many often give up thinking that there is just too much subjectivity. I’d think measuring UX can be done from another viewpoint. UX depends on many factors, not just the software interface and the ‘wow factor’ of the interface, but the entire experience in interacting with the company. An end user’s overall UX is also, therefore, influenced when they call your help desk or technical support. Some folks refer this as the front line in dealing with the customer/end user.

Current metrics in measuring service desk performance are rarely tied to software quality, but in fact, they are very related. When you have defects escaping into production, you have more phone calls, more chats, etc. And if the problems caused by the defects are severe then call lengths will go up, as well as 2nd calls for the same problem, chats to the online help desk, etc. Some measurements that come to mind include:

How to Quantify End-User Quality – Examine Their Total Experience

  • Call volumes to service desk
  • Call lengths to service desk
  • Total call minutes to service desk
  • Customer satisfaction index
  • Calls for same problem – customer calls back
  • Calls for something that worked before
  • # chats
  • response time for chats
  • chat abandons and rejects
  • # escalations between 1st, 2nd, and third tier support, the response time
  • response time for escalations between 1st, 2nd, and third tier support

Think of the wealth of data that could be analyzed. If you did a correlation between the hours put in by developers on a prior release, you would most likely find that if they did overtime, then the release was highly complex, or both, and therefore call volumes would increase. This not only helps you to quantify end user quality, but also the cost of quality, or lack thereof.