During our agile webinar, we had many questions we couldn’t get to. Here’s another one on UAT (User Acceptance Testing)

Question: Assume an UAT (User Acceptance Team) where they do not have separate user story/functional requirement as such. Functional points are designed in such a way that delivery is done in modules. Groups of modules that deliver functions will be tested by UAT. In this case, how can we measure test coverage %? My approach is if UAT touch the user story/function points in their testing/test case design, I consider that requirement is covered for testing.

Answer: This is not really related to the agile metrics webinar, but it is a metric that most people care about. My question is why? Why do you care about test case coverage? On the one hand, assuming more test case coverage leads to better quality, which is an agile objective, that is good. More the better, leading to less defects and less rework. On the other hand, writing lots of test cases and executing them takes time, even with automation. Your method of UAT touching a story point, then the requirement is covered, may not be adequate though if the goal is to make sure the feature/function/module is delivered with high quality. The word ‘touches’ is a little undefined. If you mean that it thoroughly goes through all the end points and negative cases that a user would do in real life, then yes, consider the requirement covered. Otherwise, perhaps not complete enough.

Measuring coverage for user acceptance testing is very difficult. Sometimes if you do measure it precisely, it may not be worth the effort. It depends on what you are going to do with the information, and is there other types of metrics/information that would be easier to get and more meaningful?