Defect reporting is one of the most important QA activities in software testing. Sure, we devise test strategies and plans, and test cases, but defects are the main ‘product’ that we produce that people can see as the result of our work. But what constitutes a ‘good’ defect? What are these elements’ definitions? I’ve been working in QA for 7 years now, so I thought I’d share some of our best practices:
A short sentence to describe the issue: What is the issue? Where does it happen? Don’t put too many words here since many Defect Tracking Systems (DTS) usually have a text length limitation.
Example: Error occurs after clicking “Load” button in home page.
- Test ENV/Configuration (environment)
These fields should be predefined in a drop down list in the DTS (usually configured by the DTS administrator) and you just need to select which one is appropriate. Fields may include: OS, Browser, DB, App Server, etc.
- Severity and Priority
Choose the correct one according to predefined criteria for the project. This may vary according to the standards of your company or your customer.
Below is the standard we use at XBOSoft:
– Severity: is a magnitude of bug impact to application, it shows how serious the defect is.
Categories of Severity: Blocker, Critical, Major, Normal, Minor
Blocker: system crash, functionality missing etc.
Critical: data loss; security issue etc.
Major: website hangs; script error etc.
Normal: UI issue; small function issue etc.
Minor: cosmetic problem.
– Priority: is an indicator of bug affect to company business. It shows how quickly developer needs to fix the bug.
Categories of Priority: P1; P2; P3; P4
P1-Urgent: defects are urgent issues, need to be fixed immediately.
P2-High: defects must be resolved in this release.
P3-Medium: defects would like to be fixed, but won’t hold shipment for them.
P4-Low: defects are not as strong as desirable, can be deferred to next release.
Example1: A high Priority and low Severity defect
Suppose that the company logo is not properly displayed or misspelled in home page.
Example2: A low Priority and high Severity defect
Suppose that application is crash whenever a user enters 1500+ characters in a name test box in which it should accept no more than 60 characters.
- Steps to Reproduce
– Each step should be a short sentence
– Each step should have a step number to illustrate sequence of actions.
– Each sentence should be started with a verb
1. Select “File” from menu bar
2. Click on “Open” button from the pop up dialog box.
- Actual Result
Normally ONLY use one short sentence to describe what the issue is.
Actual Result is a continuation of the steps, so there is no need to put steps in it. Besides this, the defect summary normally contains information on where the defect occurs, so don’t put too much information in Actual Result except the issue description.
Example: An error message “Unknown error” pops up.
- Expected Result
Normally use ONLY one short sentence to describe the expected result.
Example: Load File dialog box pops up.
- Additional Information/Comment
– A short description which can help developer understand the defect better
– Log file, testing data file or what else can help developer to fix the defect
– Other environments where you have not seen the defect. For instance, if the defect occurs in MAC OS 10.6.1, but not in 10.7, then describe this. Or if the defect occurs in DB2 but not in Oracle 10, etc. This kind of investigative work will save lots of time going back and forth with emails between QA and Development.
An well-written, complete, and accurate defect report can save both QA and developer’s time, and makes the QA process much more efficient while making everyone’s job easier and more effective. As QA, its our ‘product’ and provides easily recognizable value, so do the best you can.