Look for the Gorilla in the Software Testing Process

We like to think careful process keeps us safe. In reality, careful process can narrow our attention so much that the obvious slips by. That is the lesson of the “invisible gorilla” study, where people focused on a counting task failed to see a person in a gorilla suit walk through the scene. In testing, the gorilla is the defect that sits just outside the plan. If we only look where we expect problems to be, we miss the issues that users actually feel.

This article looks at how cognitive biases shape everyday testing decisions and what teams can do to widen their field of view. The aim is not to shame human judgment. The aim is to design the work so that judgment is supported by habits that catch what tunnel vision would miss.

Gerie Owen’s path to bias-aware testing

Gerie Owen did not set out to be a tester. Early on, she followed the test cases she was given and learned the craft on the job, including the chaotic period around Y2K. Over time she noticed a pattern that will feel familiar to many teams. Even strong process and disciplined execution could leave blind spots. Defects that mattered to users were not always the ones spelled out in the requirements. Learning about inattentional blindness helped connect the dots. When attention is locked on a demanding task, you can fail to notice what is right in front of you. That insight reshaped how Gerie approaches planning and review, and it led to her talk, “How Did I Miss That Bug? Managing Cognitive Bias in Testing,” which has resonated with many testers.

The lesson is not that checklists and scripts are bad. The lesson is that scripts, by design, focus attention. Focus is useful until it hides risk. Teams that acknowledge this tension make better choices about where to direct human attention and when to step back and ask different questions.

What the “gorilla” teaches testers

The invisible gorilla is not a parlor trick. It is a demonstration of how normal attention works. You can be competent, alert, and motivated, and still miss a clear event if your task demands narrow focus. In a sprint, that looks like proving acceptance criteria while an important failure mode sits one click away on a path no one planned to walk. In a release review, it looks like paying more attention to a green dashboard than to the one odd timeout that keeps showing up on a low traffic route. In planning, it looks like writing tests that confirm a feature behaves as expected, while skipping probes that would try to break the model you hold in your head.

Three recurring biases multiply this effect. Inattentional blindness narrows what we see when we focus hard. Anchoring makes first information powerful, so early green results pull later judgment toward optimism. Confirmation bias pushes us to look for evidence that fits our current story and to explain away the rest. These are not character flaws. They are common shortcuts. The fix is to expect them and to build simple practices that pull us back toward reality.

From awareness to practice

Awareness matters only if it changes what we do. The most reliable countermeasures are small routines that fit into normal work.

Start by asking better questions in design and planning. Move beyond “Does it meet the requirement” to “How will this fail in the real world” and “What would prove my assumption wrong.” Treat risk as the first anchor. Areas that intersect money, safety, privacy, or reputation deserve extra attention every cycle, even when they have been quiet. If you anchor on risk first, later adjustments tend to be proportional rather than convenient.

Balance the test mix on purpose. Automation is excellent at repeating checks and watching known signals across many environments. Human-led exploration is excellent at noticing the unexpected and following clues. Give exploratory work a simple structure so it stays accountable. Write a one sentence charter for each session. Time box the work. Keep short notes about what you tried and what you learned. Hold a brief debrief to agree on what changes. This turns roaming into discovery that the whole team can use.

Rotate perspective to break stale assumptions. Swap datasets or environments between testers for a day. Ask someone outside the team to walk a flow like a real user and talk out loud as they go. Review each other’s notes with one prompt: what did I assume. Small perspective shifts create new anchors and highlight paths that routine misses.

Make evidence hard to ignore. A green pipeline is not the same as a healthy product. Track a small set of signals that tie to customer impact, such as defects by area, escaped defects, mean time to detect, and issues found during exploration. Put these next to pass and fail counts so the story is complete at a glance.

Close the loop with short retrospectives that look for thinking traps. Do not hunt for culprits. Hunt for the moments where attention narrowed, where the first green anchored judgment, or where a belief went unchallenged. Capture one or two changes to try next time. Incremental improvements compound.

The usual suspects, in practical terms

Inattentional blindness shows up when testers work through long checklists under time pressure. The cure is not more willpower. The cure is to preserve a slice of time for exploration in areas that matter most and to vary paths and data so attention widens.

Anchoring appears when early results look good and set the tone. A few fast passes at the start of a cycle can bias the team toward optimism. Counter it by writing down up front what would change your mind. If a later signal matches the trigger, you do not have to argue with your past self.

Confirmation bias is subtle. Once we believe a feature is solid, we write checks that demonstrate it, and we explain odd failures as test environment noise. The counter is to design disconfirming tests on purpose. Pair testing helps here. One person drives while the other asks, “What would prove us wrong.” The point is not to be negative. The point is to stress the current story of how the system behaves.

A short scenario that many teams will recognize

A team that handles payments prepares a minor release. Early smoke tests pass. Performance baselines look steady. Automation is green on the paths in scope. Manual checks focus on acceptance criteria because the date is firm. Logs show a slow query during a rarely used path. It is dismissed as noise. The release goes out. Under a narrow pattern of high value transactions, the slow path becomes a failure. The postmortem is full of statements like “we saw a hint of this” and “it looked like an environment issue.”

Now replay the sprint with small countermeasures. At kickoff, the team writes a one page decision log that captures what they believe, how sure they are, and what would change their minds. They run a 30 minute premortem and list the three most plausible reasons the release could fail. They schedule two short exploratory sessions with charters that focus on high value edge cases and retry behavior. When the slow query appears, it matches a premortem risk. The team updates its confidence, holds the release for a day, and investigates. The bottleneck is found and fixed. The weekend is quiet.

The difference is not heroic effort. It is permission and structure to widen attention and to update when evidence changes.

Becoming a test advocate inside your company

A test advocate is not a person who argues for more testing in the abstract. A test advocate is a person who shines a light on how the team makes decisions under uncertainty and who helps build habits that turn surprises into learning. That advocacy looks practical. Tie risk language to customer impact so leaders see why it matters. Keep artifacts short and readable so they get used. Invite cross functional voices so anchors and assumptions get fresh air. Praise good reversals the way you praise good launches. When leaders change their minds in public with clear reasoning, they set a tone that makes it safe for everyone else to do the same.

If your context is regulated or safety critical, make these practices visible and auditable. Store session charters, test evidence, and risk calls next to the code and the automated checks. Write them in plain language. Explain not just what you decided but why. Audits go faster when your reasoning is easy to follow. Trust grows when partners can see how you weigh evidence.

What Gerie’s talk covers and why it lands

Gerie Owen’s “How Did I Miss That Bug” connects the psychology to daily practice. She shows how inattentional blindness, anchoring, and confirmation bias appear in real projects. More important, she shows repeatable ways to counter each one. Testers leave with ideas they can try the same day without new tools or heavy process. Managers leave with a clearer view of how to support discovery without slowing delivery. The draw is not theory alone. It is seeing how small, steady habits change outcomes.

Frequently asked questions, answered briefly

What is cognitive bias in testing and why does it matter?

It is a predictable shortcut in human judgment that can hide risk. In testing, it affects what we notice, how we size work, and when we say we are done. If you do not expect bias, you will be surprised by it. If you do expect it, you can design around it.

How do I know if bias is affecting my process?

Look for patterns. If planning conversations sound the same each sprint, an early estimate may be anchoring scope. If odd failures are explained away as environment noise, confirmation bias may be at work. If exploratory notes repeat the same paths while users report issues off those paths, inattentional blindness may be narrowing attention.

What practical steps help right now?

Add two short exploratory sessions to each iteration with clear charters. Write down change your mind triggers before the cycle starts. Hold a five minute debrief that looks for thinking traps, not culprits. Track one or two metrics tied to customer impact so evidence stays in view.