JMeter Performance Testing

Performance testing is more important now than ever!

Some studies have shown that users of mobile and/or web apps can detect differences in response times of 250 milliseconds or more. And that the “slower” the app, the less likely users are to keep using it. No one wants to put up with a slow performing, an unreliable site in cases of purchasing, online test taking, bill payment, or simply browsing for information. The internet puts real meaning to the old saying “easy come, easy go”. It is easier to lose users than gain them and performance is a key factor.

User Expectation Chart

User Expectations for Mobile Websites (1)

The thing about “performance” is that measuring it is not simple nor straightforward. The rapid rise in mobile smartphone features, functions, and usage combined with the nascent growth of the Internet of Things (IoT), has resulted in complex systems that impact performance in complicated ways. In some cases, sluggish response times may arise due to exposure to latencies that exist outside the bounds of the local applications. In other cases, it may simply be a result of large numbers of simultaneous users.

In the profession of software QA & Testing, performance testing is no longer a simple checkpoint before going live. It is a comprehensive and structured process to evaluate both mobile and web applications to ensure they deliver a great user experience under complex, challenging, and dynamic conditions. Load testing and stress testing of mobile and web applications should be an integral part of any launch plan.

Apache JMeter: XBOSoft’s Preferred Tool for Structured Performance Testing

XBOSoft decided to standardize on Apache JMeter performance testing because it is highly regarded, widely accepted, and supports all the functions required for structured performance testing. It has been around for almost 20 years, possesses a very active development community, and as it is written in Java, it is considered cross-platform. While JMeter by itself possesses useful functions, a large number of plugins are available to greatly extend its range and capabilities. With these, you can run JMeter, from test function to output reports. Additionally, JMeter web performance testing can test against a variety of interfaces, including HTTP, REST APIs, and others.

There are many benefits offered by open source software besides a low price. Because there are many developers constantly refining open source software, issues with security and quality can surface quickly and be addressed quickly. Open source applications can also be tailored in flexible ways as required by the business; another benefit not easily enjoyed by those using proprietary solutions. Some key advantages promoted by Apache JMeter are:

  • Open source license. JMeter is free giving developers license to create and modify without restriction.
  • Friendly GUI. JMeter is easy to use.
  • Platform independent. JMeter is a pure Java desktop application, so it can run on multiple platforms.
  • Full multithreading framework. JMeter can be used for concurrent and simultaneous sampling of different functions by a separate thread group.
  • Easy installation.
  • Highly Extensible. You can write your own tests.
  • Support multi-protocol. JMeter supports web application testing and all basic protocols such as HTTP, JDBC, LDAP, SOAP, JMS, and FTP.
  • Record & Playback. User activity can be simulated on the browser in web applications.

What are the Major Challenges in Using Apache JMeter?

Setting up Apache JMeter performance testing into a usable configuration can be tricky. Because JMeter is not a native application per se, a Java environment must be configured prior to installing JMeter. Plugins need to also be downloaded and installed. And in order to handle large numbers of simultaneous Virtual Users (Vusers), JMeter will need to be configured in a primary/replica configuration.

Perhaps the main challenge with using JMeter is not getting it operationally running, but prioritizing what JMeter will test and be analyzing results data to determine actionable improvements. You can’t test everything. A performance test plan using JMeter is a great starting point. Prioritizing tests, through inputs from analytics for how the program is being used, customer complaints, sensitivity to the function under test (e.g., financial transactions), are critical parameters that require a high degree of expertise to set.

Performance Prioritization Matrix

Figure 1 – Performance Prioritization Matrix Example

XBO has experience in addressing all the “big challenge” issues when load testing and stress testing with JMeter. Over the years we have worked through them to create JMeter services that can be configured locally on a client’s private servers, or across dozens of geographically distributed JMeter cloud-based installations simultaneously driving hundreds of thousands of Vusers, while using effective configurations to capture the “right” set up.

Performance Prioritization Graph

Figure 2. JMeter – Stepping Thread Group – Response Results  Example

What This Means For You

Take advantage of XBOSoft’s Apache JMeter performance testing expertise:

  1. Get a Performance Test Plan Using JMeter, including an automated performance testing framework that lasts and will grow with you as your needs change and grow.
  2. Take advantage of the many benefits of open source software and implement a JMeter-based performance testing framework with far-reaching support.
  3. Get quick start-up benefits with XBOSoft, with years of experience under our belts, in solving complex JMeter issues and adapting to many customers’ requirements.
  4. Rapidly develop benchmarks on your application’s performance to see where your bottlenecks are and be able to prioritize your performance improvements.

Next Steps: To find out how XBOSoft’s JMeter performance testing services can find bottlenecks before your customers do, contact us.

Does JMeter web performance testing and the subtle differences between load testing and stress testing pose challenges? To learn more about JMeter, download our “An Introduction to Performance Testing Using JMeter” white paper. From building a Performance Test Plan Using JMeter or simply guidelines on how to run JMeter more effectively, we have you covered.

(1) “What I saw at PNWPHP 2016 – Concrete5.” 20 Sep. 2016, Accessed 9 May. 2017.