Cloud communications software provider Mitel was experiencing various difficulties with new software releases due to lack of sufficient testers on staff. XBOSoft was hired to provide the needed manpower. Over time, XBOSoft's range of services has expanded, with ever greater satisfaction on the part of the customer.
If we really want to increase our testing capabilities, then we should be analyzing our ways of thinking, and identifying the strengths and weaknesses in our thinking. We all have cognitive biases we’re not consciously aware of. Why do you find certain types of defects and not others? Why are your estimates always low, and therefore forcing you to work overtime? How do our biases take shape in the agile testing process? Is it different from what happens in more traditional processes?
Cognitive biases such as inattentional blindness and anchoring can cause testers to make missteps during the testing process that can have disastrous effects.
XBOSoft client AKVA rates our services very highly. In this Q & A with Eivind Brendryen, Product Manager Farming at AKVA group ASA, he speaks about the quality of the services and the value the company places on the AKVA/XBOSoft partnership.
When XBOSoft founder and CEO Philip Lew talks about the importance of collaborative testing with the company’s clients, he’s speaking not only about testing, but also about something that he practices outside the business arena.
One powerful example: His volunteer work with the Pacific Northwest Software Quality Conference. Known by its members as PNSQC, the organization has been hosting conferences devoted to quality and testing since 1983. Phil has been among its most active volunteers in recent years.
Phil Lew has served on the board and the social media committee, made presentations, and identified keynote speakers to support the program committee’s efforts. He’s made videos and done webinars to promote the conference, among his many other activities to support PNSQC.
Agile Partners - Melissa, Phil and Drake
A happy tester is a more productive and effective tester, so how do you make sure your testers are happy with their jobs? Good salaries with great benefits and a lively work environment only go so far. Testers want job satisfaction – they want to feel like they are part of something and that their contributions are appreciated. If testers feel like their company invests in them, they will invest in the company. Here are a few tips for keeping your testers happy and making sure they feel like they’re doing more than just working for a paycheck.
Agile software development methods have become more popular in recent years as development cycles shorten and competitive pressures mount to deliver quicker. Unfortunately Agile Technical Debt tends to pile up faster than we can pay it back. You see, with agile, the objective is to deliver working software in a short time frame with each iteration ‘working’ for the customer. Sometimes we get defects, yet we have ‘working’ software. We all want minimum overhead and value “working software over documentation”. Unfortunately, with each iteration, many times we can’t fix the defects and have to put it into the backlog. Any work item in backlog should be described in detail enough for those that need to work on it.
Increase Test Effectiveness By Working Together-Extreme Testing Sitting in the same chair? Well, you don't have to be that Extreme!
One of the main methodologies in agile is extreme programming where programming is done in pairs with extensive peer review. Extreme Testing by using pair testing, not to be confused with Pairwise Testing, is a cousin of Extreme Programming developed at XBOSoft, from the testing point of view. In our own software testing practice, we have used Extreme Testing methods to increase test effectiveness and as a great way to get testers to collaborate in a purposeful way. Within Extreme Testing, we have a few non-mutually exclusive practices:
I'm in a lot of agile planning meetings and as a major part of the meeting, we need to determine the amount of work for a story. Estimating agile story points is actually one of the major parts of the meeting, deciding how much work there is for us to do! We often use Fibonacci numbers and it seems to work well, but I never knew why, so I decided to do a little searching and this is what I found.
In math, the Fibonacci sequence follows the following integer sequence:
1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144... Can you guess the next one?
We had a question come in during one of our webinars on test cases, and thought it deserved more thought than a simple answer within the webinar itself. The question was, "In reference to writing unit tests for requirements, requirements should be testable, but why not write the test when you write the requirement. Is that over bearing?"
In a recent webinar that I moderated with Ken Pugh, we had a great discussion on acceptance tests. One of the points Ken made was that we all want to move testing forward. A no brainer, right? However, why don’t we do it? Maybe we just don’t know how?
Ken made the point of testing requirements. That finding and fixing requirements defects is much cheaper than finding and fixing defects later in production. He then put up the classic exponential curve showing how steep and expensive fixes can be when they are needed for defects found in production. However, I pointed out in reference to the graph that this maybe doesn't apply anymore. . .
There were still a few remaining questions from our webinar with Srilu Pinjala in July on Test Case Development - So You Think You Can Write A Test Case that we wanted to address. It was Srilu's opinion that test cases should be very detailed and document the functionality of the software. Her view was that this way test cases can then be used by anyone and at a later date. She coined the phrase Test Like a Robot which I found interesting. Should one really want to be like a robot?
From our webinar with Srilu on "You think you know how to write a test case", we had several questions. With regard to agile test cases , "What do you do in the mean time between development and when the app/software is ready? Is it not advisable to write the test cases based on the product specification instead of waiting for the software to be ready?
I had a chat with Srilu about this, and here is what she says. . .
I've been invited to give a talk on Pairwise Testing Test Case Design in Atlanta on August 11 for the Atlanta Quality Assurance Association. My preparation, though, involves much more than just explaining pairwise testing. When using pairwise testing as a combinatorial method for test case selection, you need to think about much more than choosing what tool to use or how it works, but rather when, where and most importantly, WHY? Why choose pairwise versus 3-wise, or 4-wise? First I had to think of the Why's.
Implementing Agile QA depends on the environment, people, their skills and personalities and various other factors so we didn't title this 'Best" practices but good. This means that our goal here is to list out some practices that deserve consideration when implementing agile QA. There should be an information sharing system to document the latest information with regard to the user stories, tasks, defects, decisions made. There should be an issue tracking system (i.e. JIRA) to document processes and implement workflows. QA should understand the business background and objectives as much as possible in order to start testing from the user [...]
Many of our new clients are in the process of implementing agile and many of them have no experience in agile. They hear a lot about it, read many papers and articles and are ready to dive in, but we usually recommend a staggered or gradual approach. For those implementing agile who are used to waterfall, the most benefitial approach is to try to borrow some practices from agile and implement those pieces first. Test automation is a key part of agile, The QA team can help write lots of tests (after code is developed, in a waterfall fashion), including unit, integration and [...]
I’ve been working at XBOSoft for almost 7 years. I started as a tester and now I’m a project manager. Through the years, I’ve learned a many lessons on testing project management and how to manage projects and people. My Top 3 Most Important Things When Managing a Project are:
For defect verification, when developers are examining defects, they often complain that issues cannot be reproduced. There are many possible explanations and ways to avoid this. Let’s analyze some of the most common reasons.
As a project manager in XBOSoft, I focus on satisfying our clients which is dependent on how well I manage our testing team. Test team management requires that I communicate with team members frequently every day. In order to be able to function as an efficient team, I try my best to develop effective team communication. The five most common and effective ways we use to communicate are: 1. Team meetings 2. Email 3. IM tools 4. Team collaboration tools 5. Face to face/1-1 talks Team Meetings: Meeting in a conference room with team members directly can deliver information to all members at the same time, and it also helps to collect different ideas from different team members.
Our clients often ask “How and when the real software testing activities can be started?”. My answer has always been the same: “Once the Software Test Plan is DONE and it gets approved by you!” The Software Test Plan is one of the most important documents for a test project and is usually the first deliverable to our clients. Only after the test plan is approved will our Testing team start real testing activities. From my experience, a Software Test Plan should contain the following information at minimum: Test Scope Documentation Test Methodology Schedule Test Entry/Exit Criteria Test Tools
Defect tracking workflow is the life cycle of a defect. It describes the states of the defect from when it is created to when it is closed. There are two main defect tracking workflow models: 1. Identify defect ONLY with “State”; 2. Identify defect with both “State” and “Resolution”. You can set up a defect tracking workflow system with either of them, but which one is best for you? Let’s set up a defect tracking workflow with the most common defect states: New Open Reopen Fixed Invalid
For agile testing, the more testers know, the better. With more relevant information and knowledge, agile testers can be sensitive to potential issues and therefore be able to provide more efficient feedback on an agile project. Although implicit, most agile testers serve to bridge the product team’s expectation and actual development work. In this sense, agile testers play a critical role, especially when they possess valuable knowledge and skills to understand developers and product owners from different points of view. But to do this requires skills that are not common to many ‘traditional’ testers. What do these ‘agile testing’ skills include?
More and more companies are now using Agile as their development methodology. This requires that test cases be executed, completed in a short time, and repeated very often. To reach these objectives, automated testing is a must.
There are many ways to setup your automation testing system. The purpose of this blog is not to compare andconclude which one is the best; but mainly to show an example of creating and executing JSAS test automation, one of the automation platforms that we’ve been using lately. .
First, what’s “JSAS”?
One of the main concepts depicted in the Agile Manifesto is close communication among team members. However, in the real world, sometimes we have to face the situation that the whole agile team is split into several sub teams that may not be in the same location or time zone. This situation usually happens with global companies or when some work is outsourced to a services provider that may be in another country. As a specialized software testing company, we often have the situation where our customer develops their software while depending on us to be responsible for the quality assurance as a remote agile testing team. With agile development methods now becoming more popular, we have had to adapt from a waterfall situation where we wait for a software build based on fixed time periods to a‘remote agile testing’model. To do this, we must have a very good plan. But even before starting the project, we think about a few soft factors that have a significant impact on success but are often forgotten. These include:
Why do project managers often worry about the # of testers, and the ratio of developers to testers. It follows that how much functionality there is to test is determined by Why do project managers often worry about the ratio of the number of testers to developers? It follows that how much functionality there is to test is determined by the developers and what they produce but applying a simple ratio is simply not the answer. I’ve heard that the agile development paradigm requires more developers than testers. In my opinion, this is far from the truth. More developers may create even more concerns and increase the possibility of breaking the application if agile is not implemented correctly. So, what determines the number of testers needed? Over the years, we’ve developed a methodology to gauge the testing effort required based on several factors.
Test Team Management - Top 5 Interpersonal Skills for a Project Manager At XBOSoft, we have a well-defined process for test team management. However, sometimes there are many non-defined skills need to lead a team to success. As a project manager for more than 5 years at XBOSoft, to satisfy our clients I know that the most important objective for a team is to complete the project task with high quality and efficiency. As a project manager, my responsibility is to lead and manage the whole team to achieve this goal. However, I am always thinking about what interpersonal skills should I have besides technical skills? From my experience, the most important 5 interpersonal skills are: • Leadership • Communication • Motivation
We are happy to announce that March 20 at 10 AM EST we are holding an agile testing webinar with two veterans in the field: Jared Richardson and Philip Lew. Jared and Phil will discuss the changes needed for QA and Testing when working in an agile environment. Agile Testing Topics covered include: - Agile development trends - How to test 'agile' - How to implement scrum - Typical scrum testing bottlenecks and how to solve them - Testing agile requirements - Agile test metrics For Jared Richardson & Philip Lew Jared Richardson Jared is a software consultant. [...]
Reducing time on testing is all about effectiveness, not only for the testing process, but the entire development process. The further upstream that defects are found and eliminated, that means less defects found in testing or less defects to be found in testing. Therefore, by deduction, less time spent in testing. In terms of finding defects further upstream, most organizations don’t think of the term ‘defects’ until there is a final product to find fault in, but the term defect can be used for any mistake in the process that would lead to a defect in the end product. This means
Good living habits keep you healthy; good test management habits make your testing team strong and ensure consistent success. I’ve led many teams - from 2 or 3 to 30 or 40 people. The following are the habits I always insist on and they keep my projects running smoothly so far.
Improve Software Quality - Find Out How In This Webinar October 25 2012 - We held a webinar on Quality Process Assessment (QPA), our software quality improvement service that takes into account how to improve software quality from the viewpoint of the entire life cycle of your software product including phases that are often overlooked such as sales, marketing, customer service. Software quality is not a spice you add at the end. It must be baked in from the beginning! Enjoy the recording! Jan
October 25 at 10 AM EST we are holding a webinar in which we explain how XBOSoft’s Quality Process Assessment (QPA) can help you prevent defects throughout your software products’ entire life cycle, above and beyond development and testing. We show how to examine your software product's lifecycle and how each phase impacts your customers' view of the product's quality. This includes phases such as: - Sales & Marketing-Product/Project Sponsor (intra-Company software) - Product Management (Requirements) - Development - Quality Assurance - Customer Service/Tech Support In this webinar consultants from XBOSoft will share with you their first-hand experience. You'll learn: - [...]
Defect reporting is one of the most important QA activities in software testing. Sure, we devise test strategies and plans, and test cases, but defects are the main ‘product’ that we produce that people can see as the result of our work. But what constitutes a ‘good’ defect? What are these elements’ definitions? I’ve been working in QA for 7 years now, so I thought I’d share some of our best practices: Summary/Subject/Title A short sentence to describe the issue: What is the issue? Where does it happen? Don’t put too many words here since many Defect Tracking Systems (DTS) usually have a text length limitation. Example: Error occurs after clicking "Load" button in home page.
Last time I wrote about interviewing testers and assessing their LoadRunner skills. When assembling a team, you may need testers with LoadRunner skills, but you also need to have a balanced team. Putting together a test team effectively depends on the type of project and skills required, but in general, we like to have a blend of people. When we put together a team, we think of balancing not only skills but personality traits:
Nowadays, many companies consider integrating TPI into their testing processes. However the questions of how much does a TPI effort cost and will these costs exceed its benefits have been raised by many practitioners. This article discusses and elaborates TPI’s benefits and costs, so that you can decide whether TPI is justified. Most of the time, a TPI effort is triggered by one for more of the following three testing issues: Testing takes too long Testing is too expensive False expectation between developers and testers The primary goal for TPI is to improve the product’s quality through making the testing [...]
We do quite a few automated testing projects with QTP, one of the most popular automated testing tools. As such, as a project manager, I often interview candidates where many say they have QTP “skills” or have QTP “proficiency”. However, it's prudent to dig a little deeper...
In most testing projects, we need to setup and manage many test environments - from Windows 2000 Advanced Server to Windows Server 2008 R2, from Redhat Enterprise 5 to Solaris 11, from Microsoft SQL Server 2005 to Oracle 11g R2. This is required because testers frequently switch among them to verify and validate the quality of the products across multiple platforms. We’ve found that setting up and managing many test configurations can be accomplished well with VMWARE Esxi. VMWare Esxi, now renamed as "VMware vSphere Hypervisor", is a popular virtualization solution and has many benefits:
