This blog category includes all the posts related to test process improvement. Its parent is Process Improvement.

Quality is Key at SofTec Asia Conference

At the beginning of August, Our CEO Philip Lew spoke at the Softec Asia 2017 conference in Kuala Lumpar, Malaysia. With the theme "Testing As A Service," speakers tackled the topic with knowledge in all aspects of improving software quality. Here is a visual look at what some of the speakers focused on in their presentations.

Softec Asia 2017 theme Testing as a Service.

Testing Accounting Software – Account Reconciliation

XBOSoft Testing Accounting Software - Account ReconciliationThe complexity of account reconciliation software places special demands on the testers of account reconciliation software. The primary challenge: There are two types of accounts to be reconciled, each with its own unique characteristics.

3 Fatal Flaws of Project and Program Frameworks

QA expert and author Johanna Rothman asks in a recent article whether managers in agile testing should "scale" agile to help multiple teams deliver products. Her answer: an emphatic "No." Why?

Johanna Rothman

"Scaling process leads to bloat. Instead of scaling process, scale collaboration."

Increase Test Effectiveness Through Extreme Testing

Increase Test Effectiveness  - Extreme Testing

Increase Test Effectiveness By Working Together-Extreme Testing Sitting in the same chair? Well, you don't have to be that Extreme!

One of the main methodologies in agile is extreme programming where programming is done in pairs with extensive peer review. Extreme Testing by using pair testing, not to be confused with Pairwise Testing, is a cousin of Extreme Programming developed at XBOSoft, from the testing point of view. In our own software testing practice, we have used Extreme Testing methods to increase test effectiveness and as a great way to get testers to collaborate in a purposeful way. Within Extreme Testing, we have a few non-mutually exclusive practices:

Software QA Best Practices versus Best Principles

I recently had a client ask us to assess their situation and then give them recommendations on software QA best practices that they should consider. I know that many of my colleagues would say that there is no such thing as best practices, and I agree on that. But I think there should be a new term called Best Principles, or if you don’t like the word "best," perhaps Recommended Principles. To explain a little better, let me briefly draw an analogy.

Defect Removal Effectiveness – Agile Testing Webinar Q&A

In our recent agile testing webinar, we had an outstanding question on Defect Removal Effectiveness (DRE). Query1: For DRE calculation, I believe below is the correct measure. Can you please double confirm? DRE = Defects found pre release/DFP+Defects found in prerelease Yes, your calculation is correct. Ours is actually the inverse and would be called Defect Escape Rate (wanting to keep low) instead of Defect Removal Effectiveness. Let’s do an example just to illustrate.

Agile QA – Guidelines (not BEST Practices)

Implementing Agile QA depends on the environment, people, their skills and personalities and various other factors so we didn't title this 'Best" practices but good. This means that our goal here is to list out some practices that deserve consideration when implementing agile QA. There should be an information sharing system to document the latest information with regard to the user stories, tasks, defects, decisions made.  There should be an issue tracking system (i.e. JIRA) to document processes and implement workflows. QA should understand the business background and objectives as much as possible in order to start testing from the user [...]

Software Defect Metrics – Getting Ready for Softec 2014

As I prepare for my talk in Kuala Lumpur at Softec 2014, I started thinking about our own projects at XBOSoft and the software defect metrics that we use internally to see how we are doing for our clients. There are the normal ones such as defect escape or leakage rates, and defect fix time, technical debt reduction - refactoring. But from a 'pure qa' point of view and in particular XBOSoft, we want to reduce the work for our clients while improving the quality of their software. Some of the key metrics we track include:

Using Cyclomatic Complexity Analysis to Increase Software Quality

Ensuring software quality requires much more than testing and should start much earlier in the development lifecycle. One activity that can help is cyclomatic complexity analysis. For cyclomatic complexity analysis, simply put, higher numbers indicate more complexity and are “bad” while lower numbers are “good”. Some modules and their functions may be inherently more complex to implement and therefore have a relatively high cyclomatic complexity than other modules. Cyclomatic complexity gives us a sense of how hard code may be to test, and maintain

Software Test Team Management – Collaboration and Communication in Daily Activities

As a project manager in XBOSoft, I focus on satisfying our clients which is dependent on how well I manage our testing team. Test team management requires that I communicate with team members frequently every day. In order to be able to function as an efficient team, I try my best to develop effective team communication. The five most common and effective ways we use to communicate are: 1. Team meetings 2. Email 3. IM tools 4. Team collaboration tools 5. Face to face/1-1 talks Team Meetings: Meeting in a conference room with team members directly can deliver information to all members at the same time, and it also helps to collect different ideas from different team members.

Software Quality Metrics – Top Reasons for Failure

Have you implemented a software quality metrics program? If so, then you know its not easy and that metrics programs often fail. Why? Software Quality Metrics - Top Reasons for Failure --Metrics don’t measure the right thing --Metric becomes the goal --Metrics don’t have consistent context so they are unreliable --Measurements used are not consistent --Metrics don’t answer the questions you had to begin with --Metrics have no audience --Measurements are too hard to get --Metrics have no indicators so cannot evaluate

Software Test Plan Checklist

Our clients often ask “How and when the real software testing activities can be started?”. My answer has always been the same: “Once the Software Test Plan is DONE and it gets approved by you!” The Software Test Plan is one of the most important documents for a test project and is usually the first deliverable to our clients. Only after the test plan is approved will our Testing team start real testing activities. From my experience, a Software Test Plan should contain the following information at minimum: Test Scope Documentation Test Methodology Schedule Test Entry/Exit Criteria Test Tools

Defect Tracking Workflow – Comparing Two Popular Workflow Models

Defect tracking workflow is the life cycle of a defect. It describes the states of the defect from when it is created to when it is closed. There are two main defect tracking workflow models: 1. Identify defect ONLY with “State”; 2. Identify defect with both “State” and “Resolution”. You can set up a defect tracking workflow system with either of them, but which one is best for you? Let’s set up a defect tracking workflow with the most common defect states: New Open Reopen Fixed Invalid

Agile Testing with Jared Richardson and Philip Lew – Webinar March 20 2013

We are happy to announce that March 20 at 10 AM EST we are holding an agile testing webinar with two veterans in the field: Jared Richardson and Philip Lew. Jared and Phil will discuss the changes needed for QA and Testing when working in an agile environment. Agile Testing Topics covered include: - Agile development trends - How to test 'agile' - How to implement scrum - Typical scrum testing bottlenecks and how to solve them - Testing agile requirements - Agile test metrics For Jared Richardson & Philip Lew Jared Richardson Jared is a software consultant. [...]

Reduce the Time Spent on Testing – Testing Strategies

Reducing time on testing is all about effectiveness, not only for the testing process, but the entire development process. The further upstream that defects are found and eliminated, that means less defects found in testing or less defects to be found in testing. Therefore, by deduction, less time spent in testing. In terms of finding defects further upstream, most organizations don’t think of the term ‘defects’ until there is a final product to find fault in, but the term defect can be used for any mistake in the process that would lead to a defect in the end product. This means

Improve Software Quality Beyond Dev and Test-Quality Process Assessment Webinar

Improve Software Quality - Find Out How In This Webinar October 25 2012 - We held a webinar on Quality Process Assessment (QPA), our software quality improvement service that takes into account how to improve software quality from the viewpoint of the entire life cycle of your software product including phases that are often overlooked such as sales, marketing, customer service. Software quality is not a spice you add at the end. It must be baked in from the beginning! Enjoy the recording! Jan

Quality Process Assessment – How to Improve Software Quality – Webinar Announcement

October 25 at 10 AM EST we are holding a webinar in which we explain how XBOSoft’s Quality Process Assessment (QPA) can help you prevent defects throughout your software products’ entire life cycle, above and beyond development and testing. We show how to examine your software product's lifecycle and how each phase impacts your customers' view of the product's quality. This includes phases such as: - Sales & Marketing-Product/Project Sponsor (intra-Company software) - Product Management (Requirements) - Development - Quality Assurance - Customer Service/Tech Support In this webinar consultants from XBOSoft will share with you their first-hand experience. You'll learn: - [...]

Software Testing Metrics – Defect Analysis

Software quality is an abstract concept. So abstract, that standards bodies develop models to understand it better, i.e. ISO 25010. Quality models help us to crystalize and understand quality and quality requirements, both functional and non-functional, with the goal of evaluating them. The goal of testing is to determine if these requirements are met. During the course of testing, we find defects, or instances where the software does not meet requirements. Hence, in the area of software testing metrics, there has been abundant work in analyzing defects. With respect to analyzing defects, there are many flow charts detailing how defects flow back and forth to QA with changes in state (fixed, open, re-open, etc.). There are also numerous software applications (defect tracking systems and defect management systems) that help us track defects at the different phases of development and after release. However, these activities are rarely connected to metrics in such a way that is easy to analyze. Rather, there many defect metrics often listed out with definitions and calculations

Defining and Measuring Software Quality – Implementing a Metrics Program

In a nutshell, establishing and implementing a measurement program for software quality needs more than software testing related metrics. Testing metrics (like defect statistics) are detective type metrics which are very late in the product life-cycle, and also only include one part of the organization. What about metrics that could prevent the defects to begin with?

Software Test Metrics and Measurement – Progress Metrics

In my last post, I mentioned that metrics could be used for different purposes. One of them was tracking progress. Usually when we track progress, it is related to time, or other unit that indicates a schedule. Most often we use progress metrics to track planned versus actual over time. What we track depends on our role. If we are financial people, then we track money spent. But for software quality assurance, we want to track progress of such things as defects, test cases, man-hours, etc. Basically anything that is related to results or effort spent to get those results. For instance, here are a few:

Software Test Measurement – Introduction

Last time, I blogged on some general flows of defects and 'buckets' of measurements that I thought were possible. Then I thought that I needed to take a step back and think at a higher level about software testing measurement in general. Software testing measurements can help us in a few areas including: Planning: Cost estimation, training and resource planning, budgeting and scheduling Controlling: Track testing activities for progress and compliance to the plan Improving: Benchmarking processes and identifying where efforts should be focused and then measure the effects. There are many other areas that measurement can help us

Software Testing Metrics – Defect Analysis

Today, I sat down and started thinking about defects and how to analyze them. There are many standard defect metrics such as: Defects Found/Resolved Defect Severity Defects Found/Test Cases Executed The list goes on. I'm not really a 'mindmapper', but I decided to think of defects in terms of flows, where they come from, what influences them, and what happens to them.  I started doing a sketch and this is what I came up with. Flowchart of defect sources, inputs, and outputs Next I'll start analyzing each oval and see where that leads. If you have any input [...]

XBOSoft Presents at Better Software / Agile Development on Quality Process Assessment

Last week Alan and I were at SQE's Better Software / Agile Development conference in Las Vegas. We had a great time at the conference and in the city. At the conference Alan spoke on software quality and how to improve it using a Quality Process Assessment (QPA). With a QPA we focus on preventing defects through out the full software life cycle and also include evaluating senior management, sales, and customer service as a root cause of defects. More info on a QPA here.

Defect Writing Best Practice Guidelines

Defect reporting is one of the most important QA activities in software testing. Sure, we devise test strategies and plans, and test cases, but defects are the main ‘product’ that we produce that people can see as the result of our work. But what constitutes a ‘good’ defect? What are these elements’ definitions? I’ve been working in QA for 7 years now, so I thought I’d share some of our best practices: Summary/Subject/Title A short sentence to describe the issue: What is the issue? Where does it happen? Don’t put too many words here since many Defect Tracking Systems (DTS) usually have a text length limitation. Example: Error occurs after clicking "Load" button in home page.

Test Process Improvement Best Practices

The software world is changing by the minute. To test software that is constantly changing is a daunting task, especially when market share and profitability depends on quick delivery. In the rush to embark on a process improvement effort such as CMMI, TPI, TMMi, etc. it is easy to forget the essentials. Here are 3 best practices to keep in mind:

Software Quality Metrics for Quality in Use

When most people think about software quality metrics, they think about defect related metrics such as: Total defects, Defects at delivery, Mean time to defect, Defect burn rate, Mean time to failure, Mean time between failures, etc. While examining defects is a start to developing quality metrics, it’s just a start.

Test Process Improvement (TPI) – Benefits versus Costs

Nowadays, many companies consider integrating TPI into their testing processes. However the questions of how much does a TPI effort cost and will these costs exceed its benefits have been raised by many practitioners. This article discusses and elaborates TPI’s benefits and costs, so that you can decide whether TPI is justified. Most of the time, a TPI effort is triggered by one for more of the following three testing issues: Testing takes too long Testing is too expensive False expectation between developers and testers The primary goal for TPI is to improve the product’s quality through making the testing [...]

Show Buttons
Hide Buttons