There are many perspectives on quality. Some people many moons ago said it was conformance to requirements. These days, the end user perspective on quality, sometimes called the User Experience (UX), has come to the forefront as a hot topic of importance, and an element of software quality. But measuring measuring UX is a tough task. How can you measure a user’s ‘oohs’ and ‘aahs’? I’d like to approach measuring UX from another viewpoint. UX depends on many factors, not just the software interface and the ‘wow factor’ of the interface, but the entire experience in interacting with the company. An end user’s overall UX is also therefore influenced when they call your help desk or technical support. Some folks refer this as the front line in dealing with the customer/end user.

Current metrics in measuring software quality usually involve defects escaping into production as the primary measurement. Therefore when calls come in, defects are logged. If verified, then those defects are considered to have escaped, thus impacting the defect detection rate.

However, as you can see from the diagram, there are many other factors influencing the calls to technical support. These could be general questions, how to operate certain functions, how to install on a certain platform, call on an existing problem, call for the 2nd time by the same customer who didn’t get a good answer the first time, etc.

These types of calls outnumber calls for defects by 10 to 1, yet they go unmeasured with respect to software quality. Any time a user is confused about something and they call to ask how to make something work correctly and the tech support agent says, ‘oh, you need to do x, then y’, that experience significantly influences the user’s experience with the company and the product, yet it most often goes unnoticed by the developers.

In fact, there are many types of calls that come into the technical support line that should go into the ‘defect ratio’ so to speak. What other types of interactions or calls coming into the technical support center can you capture to help measure the end user perspective on quality?

If we could somehow capture these elements, we could get a better understanding of what the user’s perspective of the quality of the software we are providing them, and how to improve it.