Quality Process Assessment
Insight
XBOSoft’s Quality Process Assessment (QPA) examines the entire organization from the standpoint of quality – how do its people, processes, and use of technology and tools support the organization’s quality goals? Using our QPA methodology and framework, we uncover the root causes of issues that impact quality and provide a roadmap to application and process improvement.
The term “defect” usually applies to issues encountered by users and when the product does not meet requirements or behaves unexpectedly. However, XBOSoft’s QPA expands this to include errors that can occur throughout the lifecycle, from poorly defined requirements and unclear test cases to issues reproducing defects and marketing materials that may be misleading about capabilities.
Our Clients
Framework
Our QPA methodology considers the entire product lifecycle, recognizing that quality involves every stage from conception to customer use. We examine how each department influences subsequent stages and depends on previous ones:
- Sales & Marketing sets expectations and influences Product Management
- Product Management defines requirements that Development must implement
- Development creates code that Quality Assurance must verify
- Quality Assurance validates the product before the Product Delivery
- Product Delivery affects Customer Service and Technical Support
- Customer Service and Support impact the user’s experience with the product
- User interactions and perceptions ultimately define the quality experienced
This comprehensive view allows us to identify quality gaps throughout the process, not just in the QA department.
QPA Workflow
Our process follows three key steps:
Step 1 – Pre-Onsite Data Collection & Analysis
- Complete pre-assessment questionnaires and collect required documentation
- Confirm QPA objectives with team leads (VP of Development, QA managers, stakeholders)
- Gather diverse perspectives through carefully designed questionnaires
- Collect data from development artifacts including test cases, defects, user stories, quality statements, process definitions, and help desk tickets
- Review requirements specifications, marketing materials, and organization charts
- Analyze collected data to understand quality perspectives across the organization
Step 2 – Onsite Review & Off-Site Analysis
- Conduct 3-5 days of onsite work with stakeholders
- Perform one-on-one and small group interviews
- Observe work processes
- Review additional data
- Hold preliminary findings meetings with management
- Analyze critical performance areas:
- Work Flow Process
- Management Processes and their impact on quality
- QA Engineer Job Design and organizational alignment
- Technology Inventory
- Defect patterns with process improvement correlation
- Qualityware evaluation
Step 3 – Deliverables: Assessment Report & Quality Workshop
- Provide a comprehensive document with recommendations for immediate changes
- Present results in an executive workshop
- Map out an improvement plan
Impact
Many don’t realize the strong connection between support and the customer’s perception of quality. By closely examining help desk tickets, XBOSoft’s QPA determines pain points and provides actionable feedback with steps your organization should take earlier to prevent issues from being encountered in production.
Quality is the primary focus and goal of our assessment, leading to tangible benefits:
- Improved team productivity and efficiency
- Enhanced team management and morale
- Process improvement and insightful quality metrics
- Achievement of group performance and business goals
- Meeting budget requirements while delivering high quality
- Reduction in technical support calls and help desk tickets
In previous QPAs, we’ve uncovered significant requirements issues. In one instance, we followed up with a requirements workshop and worked closely with the client to revamp their requirements elicitation process. This helped them decrease requirements changes downstream and, therefore, shorten their development cycle, in addition to reducing defects relating to incomplete or ambiguous requirements.
Measurements
The Assessment Report provides concrete measurements and analyses:
- A comprehensive examination of QA operations with an executive readout of findings from the current situation
- Analysis of operational-development processes from product management to customer service
- Analysis of defects to identify trends, correlations, and other characteristics
- Technology analysis with recommendations
- Staffing mismatches based on current workload
- Work function review and recommendations
- Performance measurements analysis
- Risk analysis with recommendations for risk reduction
Many executives find our holistic approach and evaluation mind-boggling, as they are not used to viewing or analyzing the organization solely based on quality. Many only expect a defect analysis and defect-related metrics. Still, when we do our correlative evaluation and can tie defects directly to requirements-related metrics such as requirements volatility or correlate defects directly to customer service and tech support wait times and numbers of calls, they can see that quality or lack of quality permeates through the entire organization’s key performance indicators.
Let’s make your success our next story
Challenges
Organizations often face challenges in maintaining quality across departmental boundaries. Our assessment addresses these challenges by:
- Identifying disconnects between departments that affect quality
- Recognizing when quality issues originate outside the QA department
- Understanding how marketing expectations affect customer perceptions
- Addressing issues in requirements that lead to downstream defects
- Improving communication between development and QA
- Enhancing how customer support feedback influences product improvements
Almost always, the individuals in the organization understand the issues and challenges within their scope of responsibility, yet don’t have a clear picture of how all the pieces fit together and impact the organization’s software quality as a whole. By working closely with periphery departments, we can piece together the entire quality picture, and how it’s impacted at various levels. For example, customer service calls and their root causes can usually be traced back to earlier stages in the product development.
In one organization we worked with, their marketing department emphasized product ease of use, but that value was not a significant development team strategy. This led to a mismatch of customer/end user expectations, and what was delivered. Regardless of working functionality, the perception of quality was poor from the end user’s perspective. Isn’t this what counts? By uncovering this simple issue and recommending changes to market positioning based on the software’s strengths, we reduced calls to the customer service department by 18 percent. This sounds small, but the effects for the entire department were affected since the remaining calls had a decreased wait time of 34% leading to better customer satisfaction and service.
Experience
We’ve worked with organizations of all shapes and sizes to examine their processes and offer guidance toward long-term, sustainable improvements. Smaller organizations with development teams of under 10 and a total staff of under 50 can sometimes have the owner involved in the product development process and staff with responsibilities that cross different functions. They are usually trying to get an MVP to market as soon as possible, or they could be needed to get to the next step in quality because they’ve released a product and realize that both customer support and software quality now need more focused attention. For these types of organizations, our QPA is particularly helpful in making process recommendations where there is no progress and helping them to see the need for processes in critical areas.
For larger teams, we have worked with some organizations that have development teams of well over 200. We’ve found over the years that information is siloed across many departments, often leading to:
- Communication gaps
- Duplicate work
- Ambiguous responsibilities
- Minimal metrics to measure improvement
In these cases, many of our recommendations centered around clearing up processes to remove duplication while ensuring that metrics are installed to measure process effectiveness. By understanding the communication gaps and why they happen, we can also help the organization to reduce work in progress.
Certifications
Certified expertise, reliable results
We hold PMP, ISTQB, and ISO27001 certifications, reflecting our adherence to rigorous standards in project management, software testing, information security, and internal controls.