We’ve been working with a financial application that does account reconciliation for several years now. The financial software we test conforms to SAS70 (Statement on Auditing Standards No. 70), so all of the data and workflows are designed according SAS70. Although we aren’t accountants, certainly testing this type of software needs a more careful eye than ordinary UI testing, so I thought I’d share some of the best practices developed by our team over the last few years.
What to look out for?
Firstly, SAS70 defines the standards an auditor must employ in order to assess internal controls. This is especially important for public companies that must adhere to numerous regulations, so it’s important to understand SAS70 as it impacts data and business work flows, as well as roles and privileges of those that have access.
Best Practices
1. Testing input fields needs to be comprehensive. It should include legitimate data and invalid data. Negative testing is a must. For example, when testing an input box, if the requirement is a number, it is necessary to enter a number to test, but you also need to enter letters, spaces and other characters. This is certainly not new for testing, but very important for testing financial applications. For example, the amount and percent field should not allow non-numeric values.
2. Data integrity and accuracy is critical. So within workflows and related workflows that access the same data, you not only need to check all calculations, but also if the number contains an integer, negative, or decimal values. All need to be checked for consistency and format. For example, clients may use a currency other than USD, so the values need to be checked by calculations based on the currency rate.
3. Use the division of “equivalence classes” to improve test efficiency. Tests on data fields, for example, should be divided into equivalence classes to improve test efficiency. For instance, if a field allows only the number between 1 and 31, then numbers less than 1 or greater than 31 should not be able to be saved.
4. Use boundary value testing. For example, if a field allows only the number between 1 and 31, the numbers 1 and 31 should be invalid.
5. Repeat submission of the same transaction. Often users may do this if they are impatient so this needs to be tested for all transactions. For example, when a save operation is being performed, a loading panel is displayed to prevent users from clicking the save button again.
6. Don’t follow the normal sequence to test functions. This is classic negative testing, not only for normal functionality but also for workflows. Users sometimes do the opposite of what they are supposed to. For example, in a multi-currency environment, try something that does not make sense for the user to do.
7. Understand the logic and flow of the entire system. Execute normal test cases and workflows, and then observe for any abnormalities of output. As a simple example, delete a record to observe the impact of the sort. Check before and after deleting, to determine if the sort order is correct.
8. System security is critical. In addition to the data accuracy and workflow validity, as mentioned above, system security is critical. Make sure that you’ve mapped out a chart of what users can and cannot do with different roles and different levels of data, and test all combinations. Think like an end user, a good one that knows what they are doing, one that is a novice, as well as a possible malicious user. Create test cases according to the functional logic and workflows from these viewpoints. There are many!! But use 3 and 4 above to gain efficiencies. For example, once a reconciliation is certified, modification should not be permitted again.
9. Use many configurations. Most financial software can be configured with many parameters, i.e. currency, format, etc. Again, the combinations are endless so use a good test case design technique to get the most value and test different configurations as we have found that many defects are found in these areas. For example, users have different certification levels, so the display data shown to the user and certification rule should be based on the role definitions set for the company.
10. Increase domain knowledge. For example, in our case, the software conforms to SAS70, so we make sure we simulate all the roles and situations according to SAS70 procedures, from both the varying user roles and then the auditor role. SAS70 is a significant part of our training program along with working with the software as it supports us with doing ad-hoc and exploratory testing as well.
Leave A Comment