For defect verification, when developers are examining defects, they often complain that issues cannot be reproduced. There are many possible explanations and ways to avoid this. Let’s analyze some of the most common reasons.
- Sometimes, testers did not provide enough details. For example, an issue only occurs on IE9, but testers just mark it as IE. Then the developer attempts to duplicate using IE10, and cannot reproduce. To prevent this, the tester should describe every detail of each step and exact environment information. The developer should also pay attention to each documented detail to reproduce as well. This type of cross platform testing should be done, not because it is specifically required, but because you need to do this to help out the developer in troubleshooting.
- Some issues happen sometimes, but not every time. This situation needs investigation. For example, null pointer or array bounds cause a crash. If tester found some issues that cause a crash, they should try many times with different conditions to characterize the precise circumstances if possible.
- Different test data. For example, when the database contains 10,000 records, it will have different performance as compared to just 10 records. So preconditions like test data quantity, type, variation and anomalies needs to be carefully documented. Data verification testing is a key part of any testing even though it may not be specified.
- First time! Some issues only happen when first installed or launched initially and don’t have thereafter.
- The system is not clean. Maybe the test environment or browser has been modified or contaminated/altered by other applications. Keeping a clean test environment is necessary, for both testers and developers.
- Some required patch was not installed. For example, if an Adobe Flash update was not installed, some images may not appear.
In summary, for defect verification and avoiding “cannot reproduce” situations, good habits are required for both developers and testers especially in a cross browser environment. First, make sure that test environments and test data are consistent and documented. Second, make sure descriptions are clear and in sufficient detail. But regardless, sometimes “cannot reproduce” still occurs. Then the testers and developers need to actually work together!