JMeter Performance Testing

Performance testing is more important now than ever!

Some studies have shown that users of mobile and/or web apps can detect differences in response times of 250 milliseconds or more. And that the “slower” the app, the less likely users are to keep using it. No one wants to put up with a slow performing, an unreliable site in cases of purchasing, online test taking, bill payment, or simply browsing for information. The internet puts real meaning to the old saying “easy come, easy go”. It is easier to lose users than gain them and performance is a key factor.

User Expectation Chart

Have Questions? We’re Here to Help!

User Expectations for Mobile Websites(1)

The thing about “performance” is that measuring it is not simple nor straightforward. The rapid rise in mobile smartphone features, functions, and usage combined with the nascent growth of the Internet of Things (IoT), has resulted in complex systems that impact performance in complicated ways. In some cases, sluggish response times may arise due to exposure to latencies that exist outside the bounds of the local applications. In other cases, it may simply be a result of large numbers of simultaneous users.

In the profession of software QA & Testing, performance testing is no longer a simple checkpoint before going live. It is a comprehensive and structured process to evaluate both mobile and web applications to ensure they deliver a great user experience under complex, challenging, and dynamic conditions. Load testing and stress testing of mobile and web applications should be an integral part of any launch plan.

Apache JMeter: XBOSoft’s Preferred Tool for Structured Performance Testing

XBOSoft decided to standardize on Apache JMeter performance testing because it is highly regarded, widely accepted, and supports all the functions required for structured performance testing. It has been around for almost 20 years, possesses a very active development community, and as it is written in Java, it is considered cross-platform. While JMeter by itself possesses useful functions, a large number of plugins are available to greatly extend its range and capabilities. With these, you can run JMeter, from test function to output reports. Additionally, JMeter web performance testing can test against a variety of interfaces, including HTTP, REST APIs, and others.

There are many benefits offered by open-source software besides a low price. Because there are many developers constantly refining open-source software, issues with security and quality can surface quickly and be addressed quickly. Open-source applications can also be tailored in flexible ways as required by the business; another benefit not easily enjoyed by those using proprietary solutions. Some key advantages promoted by Apache JMeter are:

  • Open source license. JMeter is free giving developers license to create and modify without restriction.
  • Friendly GUI. JMeter is easy to use.
  • Platform independent. JMeter is a pure Java desktop application, so it can run on multiple platforms.
  • Full multithreading framework. JMeter can be used for concurrent and simultaneous sampling of different functions by a separate thread group.
  • Easy installation.
  • Highly Extensible. You can write your own tests.
  • Support multi-protocol. JMeter supports web application testing and all basic protocols such as HTTP, JDBC, LDAP, SOAP, JMS, and FTP.
  • Record & Playback. User activity can be simulated on the browser in web applications.

What are the Major Challenges in Using Apache JMeter?

Setting up Apache JMeter performance testing into a usable configuration can be tricky. Because JMeter is not a native application per se, a Java environment must be configured prior to installing JMeter. Plugins need to also be downloaded and installed. And in order to handle large numbers of simultaneous Virtual Users (Vusers), JMeter will need to be configured in a primary/replica configuration.

Perhaps the main challenge with using JMeter is not getting it operationally running, but prioritizing what JMeter will test and be analyzing results data to determine actionable improvements. You can’t test everything. A performance test plan using JMeter is a great starting point. Prioritizing tests, through inputs from analytics for how the program is being used, customer complaints, sensitivity to the function under test (e.g., financial transactions), are critical parameters that require a high degree of expertise to set.

Performance Prioritization Matrix

Figure 1 – Performance Prioritization Matrix Example

XBO has experience in addressing all the “big challenge” issues when load testing and stress testing with JMeter. Over the years we have worked through them to create JMeter services that can be configured locally on a client’s private servers, or across dozens of geographically distributed JMeter cloud-based installations simultaneously driving hundreds of thousands of Vusers, while using effective configurations to capture the “right” set up.

Performance Prioritization Graph

Figure 2. JMeter – Stepping Thread Group – Response Results  Example

What This Means For You

Take advantage of XBOSoft’s Apache JMeter performance testing expertise:

  1. Get a Performance Test Plan Using JMeter, including an automated performance testing framework that lasts and will grow with you as your needs change and grow.
  2. Take advantage of the many benefits of open source software and implement a JMeter-based performance testing framework with far-reaching support.
  3. Get quick start-up benefits with XBOSoft, with years of experience under our belts, in solving complex JMeter issues and adapting to many customers’ requirements.
  4. Rapidly develop benchmarks on your application’s performance to see where your bottlenecks are and be able to prioritize your performance improvements.

Next Steps: To find out how XBOSoft’s JMeter performance testing services can find bottlenecks before your customers do, contact us.

Does JMeter web performance testing and the subtle differences between load testing and stress testing pose challenges? To learn more about JMeter, download our “An Introduction to Performance Testing Using JMeter” white paper. From building a Performance Test Plan Using JMeter or simply guidelines on how to run JMeter more effectively, we have you covered.

(1) “What I saw at PNWPHP 2016 – Concrete5.” 20 Sep. 2016, Accessed 9 May. 2017.

What our clients have to say…

“They automated the web application testing by almost 95% within three months.”

- Lead QA, Whil

Whil. Logo

“They’re technically skilled and great at software development.”

- Director of PMO, MatrixCare

MatrixCare Logo

“They're very proactive and always follow through on tasks.”

- President, REAL Software Systems

REAL Software Systems Logo

"XBOSoft is able to see things from a different perspective and ask smart questions."

- CEO, Mobile MedSoft

Mobile Medsoft small

"Their work is very good and their attention to detail is excellent."

- Director of Engineering, Sago Mini

Sago Mini

"Our employees are always either requesting to work with
them or requesting to have more of their resources."

- Software Engineer, BlackLine

XBOSoft BlackLine case study

"XBOSoft has some of the most detailed and careful
test execution and reporting I’ve ever seen."

- DevOps Director, Bongo

Bongo White Logo

"They work hard and make the customer happy."

- Agile Coach, AKVA group Software AS

Akva Logo

“A CEO who’ll make an effort to visit you once a year to make sure you are happy with their
services is at the top of my personal list [of recommendations.]”

- Peter Menhart, Senior Lead Software Developer, Mitel

Akva Logo

“The key to any software application is to have the interface offer the flexibility that will drive
fast and accurate results. XBOSoft enables us to achieve these broad-based goals.”

- Project Manager, Adra Match

Adra Match Logo

“What I have seen so far is that you have done a great job!”

- Project Manager, AdminControl

AdminControl Logo

“In our early discussions and pilot testing program with XBOSoft, they demonstrated
impressive knowledge and experience in our type of CPM application. ”

- VP of Development, Host Analytics

Host Analytics Logo

"We have appreciated the high quality work your team
has done as well as your leadership"

- Board Member, Rightsline

Rightsline Logo

“"You've produced a good piece of work! Excellent!" ”

- Chief of Development, QuestBack

QuestBack Logo

“When our requirements change, they’re willing
to shift and adapt to what we need.”

- Software Developer, Telecommunications Company

Put our team of experts to work for you!

Put our team of experts to work for you!