Watch Out For These Mistakes When Doing Performance Testing

What Are The Top 5 Sources of Software Performance Issues

Software performance problems are difficult to find, understand and therefore fix because of today’s software complexity. Not only do we have to think about the software itself, but all of the other integrated components (hardware, software, database) and third party modules.

In this blog, we discuss five common sources of software performance problems. As part of the discussion, we hope to convey some of our experience in finding and solving them as you learn some mistakes to avoid along the way.

Code Efficiency

How can we discover a code bottleneck? Well, if we see for example, a certain transaction is slow, we can test the same transaction within different user scenarios to verify that the particular transaction is in fact a bottleneck. It’s important to note that a transaction is coding addressing a specific action. Examples of a transaction include:

  • Entering an address and saving
  • Retrieving a patient record
  • Viewing test results

Once you discover the general area or function involved, you can apply white box or static code analysis techniques to dig deeper to find any potential bottlenecks.
If you design your transactions at the right level of atomicity, then you can use black box methods along with your A/B testing hat, (not for usability but for performance) by varying one parameter at a time to deduce where the problem is. In this way, you can deduce if it is a code, database or some other problem. Whenever you go down this path, be sure to record all of your test parameters and scenarios so that you can track what was changed and the associated result. Design your test scenarios carefully with a specific discovery objective.

Data Storage and Retrieval

Accessing data for performance testing purposes presents challenges due to database complexity and size. One method is to use brute force to access the database directly. This allows you to discover the performance of certain data item types accessed within the application. Another approach is to go through the application as an end user would, writing specific user scenarios to access targeted data columns. In this way, you can identify which data fields are the slowest. Going one step further and you can modify the scenario to test 50 users accessing different data items in order to figure out which data items have slow response under a given situation. Taking another angle, you can vary the size of the database, while keeping the test scenario fixed to see if the database size influences the performance results. The key take away is that designing smart test scenarios is very important to assessing database performance.

Network Usage

If you need to find out how network latency is affecting your overall performance for users across different locations, design tests in the cloud using different servers to run the performance scripts. This will enable you to pinpoint network bottlenecks.

Hardware Efficiency

Another common source of performance issues is the server related hardware. The first step is to run a series of tests to establish a benchmark on your current hardware platform. Then, singularly change one element in your platform and observe the results. If a performance issue is noted, you can test deeper by installing probes in the hardware to measure CPU and memory usage as well.

Workflow Design

Poorly designed workflows for an application or system are a common source of performance problems. Testing these requires deeper domain knowledge of the software’s purpose so that you can determine how a real user would use the product. Your domain knowledge and corresponding tests need to be targeted towards the specific user type. For instance, if you are testing an electronic health record system, then you need to approach it from the viewpoint of a specific user, such as a doctor, or nurse, technician, etc. Start by looking at workflows that are the most commonly utilized by different users. Then select those that have the worst performance. You can then make an assessment to determine if the objective of the workflow is being achieved, and if there is a better way.

As in most performance testing exercises, finding the right answer is futile if you are addressing the wrong problem. Through this blog, we’ve illuminated some of the key problems that we’ve encountered over the years when troubleshooting and testing software application performance. In closing, what’s critical in approaching these common software performance problems is:

  • Recognizing that you must carry out well designed and controlled experiments, changing only one variable at a time.
  • Deeply understanding the product and the different user types. Otherwise your test scenarios won’t be representative of what real users do.
  • White box methods such as directly querying a database or static code analysis can also be used to test and quantify performance. While these methods can be used to supplement your analysis, we prefer testing from the perspective of what an end user will experience.

Leave a Reply

Show Buttons
Hide Buttons