Our clients often ask “How and when the real software testing activities can be started?”. My answer has always been the same: “Once the Software Test Plan is DONE and it gets approved by you!”
The Software Test Plan is one of the most important documents for a test project and is usually the first deliverable to our clients. Only after the test plan is approved will our Testing team start real testing activities. From my experience, a Software Test Plan should contain the following information at minimum:
- Test Scope
- Test Methodology
- Test Entry/Exit Criteria
- Test Tools
- Test Tracking and Status Reporting
Test Scope: Sometimes also called “Test Objective” or “Test Requirement”. This lays out:
- What features will be tested (focused on) in the application?
- What types of testing methods need to be covered during the testing?
Documentation: We usually ask our client to provide documentation like product documents, specifications and training materials. This information is listed out in Software Test Plan as a reference. It helps us to learn the product faster and therefore test more efficiently as well as find deeper domain related defects.
Test Methodology: Although many only think of development methodologies, there are also numerous methodologies available for testing software. The testing method is not necessarily dependent on the development methodology and depends on factors such as the nature of project, the product schedule, and resource availability. Below are some of the test methodologies that we use:
- Waterfall model
- V model
- Agile model
Schedule: Normally, the timing of testing activities follows the development progress. Whether testing can be started or not depends on build availability provided by the development team. Defining the Software Test Plan and testing schedule goes a lot smoother if the testing team has access to the development plan. With regard to schedule, out software test plan includes at minimum:
- When testing starts/ends
- Testing activities for each team member during this period
Test Entry/Exit Criteria: Criteria for Entry and Exit should be clearly defined for every activity in a testing project. It defines entry/exit criteria for starting, stopping, suspending and resuming test activities. We also have criteria defined for specifying when testing is complete. For example, one of the Test Exit Criteria I experienced before is, during the last 2 weeks prior to product module release date, if no new P1 (Priority1) defects were found in testing, testing on this module should be stopped and the module will be released to the product team on time successfully. Else, testing must be re-executed until this criteria is satisfied. Usually extra hours of testing / development is inevitable in this situation.
Test Tools: Tools used in test activities need to be documented/defined in the Software Test Plan. Many of our clients want specific tools used for specific purposes in order to compare past data and do reporting or because they already have expertise in that tool. For each tool, its important to document what the tool will be specifically used for as some tools have multiple functionalities. Some common test tools that we use include:
- Defect tracking system (DTS)
- Test case management system (TMS)
- Test management tool (information sharing, collaboration etc.)
- Downloading builds, FTP, etc.
- Other tools which need to be installed to support the client’s build/application
Test Tracking and Status Reporting: This includes how to track test activities and what kind of status report needs to be delivered. The reporting frequency should also be defined in this section (e.g. daily report? weekly report?). Additionally the tool used for Test Tracking and Status Reporting should be defined. We mainly do both daily reports (for short period projects) and weekly reports (for long term projects).
From my experience, the most important activity in a test plan is the Test Scope, since it is impossible to start work without a scope and goal. Sometimes a client may provide a rough test scope like “execute a round of regression testing with ABC application”. It is not a detailed one but gives a general direction on where should we go and what to do with the project. Additionally, documentation is often ignored or forgotten by the project manager when making a test plan. But I think it is a big mistake to exclude this section in the Software Test Plan since documentation such as product documentation and functional specifications can provide very useful information to the QA team.