When we start working with a client who is in the process of converting or have converted to Agile from Waterfall, one of the most common questions is “What do testers do in Agile?” The typical consulting answer is that it depends. As much as I hate to give that answer, it really does depend on what Agile recipe you are using and the skills of your team. For now, let’s take a step back and start with Agile planning. Testing should be a critical part of the planning process and two of the most important aspects I’d like to address are acceptance criteria within the user stories, and what should the primary testing tasks be within the Agile process.

Agile Acceptance Criteria

The availability of acceptance criteria in user stories at the planning/scope level is key. I can’t emphasize enough how important acceptance criteria is in planning. In order to plan, we size tickets; in order to size, we need to understand the scope (i.e. what should be done as part of that ticket and what not). To avoid over-engineering/under-engineering we use acceptance criteria and set expectations between the PO and the team and among the team itself. We make sure all tickets include acceptance criteria from both a business and technical perspective, so the entire team is engaged. We’ve found it helpful to create baseline technical acceptance criteria as well, such as unit test coverage standards, instrumentation, logging, performance, etc.

Acceptance criteria does have a bearing on when the user story can be considered completed, both from a development and testing perspective. We use acceptance criteria per user story – for us it means everything needed for that user story to be completed (a checklist) such that it can be closed after testing. The definition of “done” is the standard we have for our teams for every user story across the board (unit test in place, code reviewed, fully tested, deployed, etc.). We groom our user stories in a way that covers both development and testing, and have every ticket cover all defined aspects of development and testing. At the end, each ticket should serve a business value and getting to “done” with the ticket should serve both development and testing.

Acceptance criteria is vital to the Agile process. It’s necessary from the start of planning, through development and to the end of testing. Without the presence of acceptance criteria, teams may not have a solid basis to move forward from each user story, putting the project schedule at risk.

Agile Testing Tasks

One of the tenets of Agile is that each sprint or iteration has a velocity thereby quantifying the amount of work or delivered value within each sprint. As such, each sprint will have user stories as part of the delivered ‘working software’ for that sprint. Each sprint will have an estimated amount of work involved which is where the poker playing Agile sessions effectively solicit team members’ estimates for how much work is involved in a user story. Unfortunately, some teams often forget that all user stories should have a testing component, not just unit tests, but also acceptance tests along with many other QA related activities. Without this added element of work added to each story, many testing tasks often don’t get done because there just isn’t enough time allocated. It then becomes an afterthought which could result in your demo to the product owner at the end of the sprint being “Not done”. To avoid this, give testing activities the attention and focus they deserve.

We’ve identified the following key testing tasks that should be included in the Agile process:

  1. Infrastructure-related tasks such as tools, frameworks, and environment management. Especially with working remote no longer an exception, ensuring that test environments and frameworks are allocated consistently amongst your team is critical to avoid incompatibilities and asynchronization.
  2. Test data generation and maintenance, and data organization and validation has become an essential activity as using data consistently is necessary to ensure the dependability and consistency of test results.
  3. Non-functional testing tasks should also be included such as static analysis, complexity analysis, performance testing, and security testing. By conducting these activities as development occurs, you’ll have avoid huge ticking time bomb when you deliver your final product.
  4. Cross-functional or integration testing tasks, such as those that will cross from the new features for that sprint to old features, are essential as you must exercise the software as an end user would, not just the particular feature being delivered for that sprint.
  5. For acceptance tests, there are several sub-activities including determining priorities; considering normal, alternate, and error paths through the story; and story review, which means clearing up any ambiguities, clarifying story descriptions and user roles, and eliminating or delineating possible alternate interpretations.
  6. Any level of testing documentation as decided as part of your Agile recipe. (This should be an explicit task.) While some proponents of Agile believe in no documentation. Our philosophy is to document only what you want to remember. If you don’t care about remembering it, either as an organization or individual, or across individuals who may share a task, then don’t document it. If you use this idea in your documentation strategy, you may find that while you don’t need a long 20 page test plan, you do need some documentation. Sit down with your teammates and think carefully what you want to remember.

As you can see, there are plenty of testing activities as part of the Agile planning phase. The bigger picture of Agile is that everyone is responsible for a quality product and there is no “gatekeeper” of quality.