The BEST Guide To A Successful Agile Testing Strategy

An agile testing strategy is key if you want to increase the speed of delivery, quality, and efficiency of your software. Discover what it takes to create a successful agile testing strategy.

What is Agile testing?

Agile testing is difficult to succinctly define. Rather, testing needs to be thought of as an approach for ensuring the quality of products being developed using an Agile methodology. It’s a QA “way” of thinking when supporting an Agile-developed product. Traditionally, when software was developed following a “waterfall” methodology, QA knew what was going to be tested in advance, so they were able to plan and document accordingly for it. But for Agile, QA doesn’t know what QA activities will be required much beyond the current iteration. As Agile processes go, they need to be adaptive in nature to help a team deliver functionality on an ongoing consistent basis, meeting changing requirements and needs quickly. It doesn’t necessarily mean that the overall software development happens faster, it just means the evolving needs and requirements are met faster and in smaller pieces. To do so effectively, QA needs to adopt different strategies for testing Agile-developed software.

The goal of Agile testing is to support the continuous delivery of reliable products to the market in a shorter time. Under Agile development, code with corresponding functionality accumulates gradually with each progressive iteration as each piece of functionality or story is developed. As the code builds and various sections and components are integrated in, more test tasks are required by QA. As such, Agile testing requires testers to initiate automated regression testing to quickly validate existing and previously delivered functionality while also being in sync with development to test new features and functionality as they are added. Additionally, to ensure that features are developed with the expected behavior, QA should also be responsible for not only clearly understanding the user stories, but also providing valuable feedback for elements of functionality that are not specified or could be interpreted ambiguously.

What is Continuous Delivery?

In the process of software development, Continuous Delivery is the ability to get changes of all types—including new features, configuration changes, bug fixes, and experiments—into production, or into the hands of users, safely and quickly in a sustainable way. Depending on the industry, the term “continuous” could mean literally every time a new piece of code is checked in, or it could be as long as several weeks, depending on the organization and its environment. For example, for accounting software, you may not want to have a new release anywhere near the end or beginning of the month or quarter.

Why Continuous Delivery?

Continuous Delivery helps to achieve several important benefits:

• Faster time to market, lower costs It’s not uncommon for the integration and test/fix phase of the traditional phased software delivery lifecycle to consume weeks or even months. When teams work together to automate the build and deployment, environment provisioning, and regression testing processes, developers can incorporate integration and regression testing into their daily work and completely remove these phases, eliminating many of the fixed costs associated with the release process.
• Higher quality, better products Continuous delivery makes it efficient to work in small batches, providing feedback from users throughout the delivery lifecycle, ensuring quality is built into products and services from the beginning. Through automated tools, teams are freed to focus their effort on user research and higher-level testing activities such as exploratory testing, usability testing, and performance and security testing.
• Happier teams Research has shown continuous delivery makes releases less painful and reduces team burnout. Furthermore, when we release more frequently, software delivery teams can engage more actively with users, learn which ideas work and which don’t and see first-hand the outcomes of the work they have done. By removing the low-value painful activities associated with software delivery, we can focus on what we care about most—continuously delighting our users

Roles of the “Testers” in Agile Testing Strategy

In Agile, testers don’t have the role of just ‘checking’ what is thrown over the wall from development. Instead, think of the role of a general contractor. The general contractor must read the drawings very carefully and understand the requirements (user stories), asking for clarification when necessary. As the project progresses, the general contractor will have to make sure all the interfaces work together. The wood floor in the living room completed by one subcontractor must be at the same height as the tiles in the bathroom done by the tile expert, else you’ll trip. Likewise, in Agile testing, the ‘tester’ has to do more than test each specific functionality but how the whole product fits together.

• Testers are the voice of the customer: As an Agile tester with intimate knowledge of the user stories, Testers can view the product through the entire user path, as a real user, rather than separate parts of functionality. Additionally, if the client has changes, those become change orders in contractor speak, which means there could be choices to make:
  • Do you want to add this and subtract something else?
  • Are you aware that this will increase the cost?
  • Are you aware that this will delay the release?
• Testers bring focus: Revolving around a behavior-driven user-focused approach to bring clarification to what the stakeholder wants and to give a user perspective that the product owner may not be seeing.
• Testers are testing continuously: So that not only can defects be uncovered early and rapidly fixed, but features can be adapted and changed earlier rather than later if needed to satisfy the end-users and product owner.
• Testers ensure that regression testing is a continuous process Testers, having a thorough knowledge of the product as a whole, will do regression testing according to the sprints in a focused manner, ensuring all the old features/functionality work well with new features as they are inserted. Understanding what functionality touches other functions as a whole is critical to testing effectively in agile. Rather than doing complete regression, focused regression is required to maintain delivery cadence.

Strategies for Handling short testing cycles in Agile

To adapt a traditional testing model to Agile development, you may think just do everything faster and in shorter compressed cycles. But if you do that, you’ll go crazy because you’ll never be able to keep up. Hence the need for an Agile Testing strategy that is more than just do what you did before, but faster. When developing a strategy kept in mind the following:

• Test Early

As mentioned above, the conventional test cycle breaks down with Agile development as waiting to test at the end creates problems. Therefore, for each new feature, function, or bug-fix, test as soon as available, instead of waiting to the end of the time box. When the end of the cycle is reached, do targeted regression. Most of the bugs should have already been discovered and known. Early testing also means examining requirements before any development starts and determining their testability. If a requirement or feature is not testable, how will you know if it’s done to satisfaction?

• Automate

Everyone loves automation. Automated smoke tests can be used to cover the most essential functions of the software, but only automate if the tests can be created and executed efficiently. You don’t want to spend your time automating something that is difficult to do and possibly flaky, especially if it’s easy to test manually. Normally, you want to save your manual testing effort for new features and functions, but it’s not a steadfast rule. Once the software is ready for release, the smoke test scripts can be easily augmented to become acceptance test scripts.

• 360 Degree Testing

Traditional testers usually are tasked with functional testing, but Agile testing should also encompass skills around unit testing, static code analysis, and API testing during the development process as functionality often cannot be fully tested until the end. It’s also critical for testers to understand the requirements fully while thinking from an end-user viewpoint to conduct business-level user acceptance tests before the product finally is delivered to business end-users.

• Test Results Analysis

From the beginning of product design and requirements analysis to product beta testing and eventual release, QA testing should be carried out at each step as specified above. What’s just as important as testing itself and reporting defects, is understanding the results of the tests so that the team can improve. Each retrospective for an iteration should contain critical product quality metrics, with goals on improving them along with root cause analysis. With an understanding of where faults are happening, when, and why, then it’s possible to improve. Otherwise, you and your team get back on the treadmill with no improvement.

• Regression Test

While many teams find no time for regression testing in Agile, that depends on how you define it. Do you have time to do complete regression for every iteration? No. But what types of regression can you do? How should you define your targets for each iteration? Should testing focus on different aspects of testing and different functions of the product as the product is developed? You bet. This is the most important part of the iterative integration process, so unit regression, iteration regression, and business verification regression are needed on a targeted basis.

Go Agile but Don’t Forget an Agile Testing Strategy

Over the past several years there has been an avalanche of development teams and shops converting to the new Agile paradigm. Agile development offers meaningful engagement between stakeholders (clients, users) and coders, and when done right, better predictability of costs, delivery schedules, and product quality.

Agile development, however, does not preclude a role for QA & testing. In fact, the successful implementation of Agile testing techniques and strategies provides the critical foundation for improved product quality. From a quality assurance perspective, this includes testing as soon as code is checked-in, providing immediate feedback to developers, and continuous iteration regression through automation to identify defects, basic user issues, and functional issues as early in the development cycle as possible.

Agile testing is not a clearly defined process, but rather an approach for ensuring the delivery of high-quality products on schedule. It’s a QA “way” of thinking for supporting Agile development. While each project may have a specific Agile testing strategy associated with it, in almost every case, continuous delivery and automated testing will be part of the program.