A QA process is a set of gates, not stages

Seven-stage QA process guides describe work a small team already does. What stops bugs reaching customers is the gates between stages, each with a named owner.

Five floating smoked-glass gate panes recede across black, with the middle pane lit orange

Guides to a QA process draw the same seven boxes: requirements, planning, design, execution, reporting, regression, release. A small team already does all seven without the diagram, and its bugs still reach customers. The diagram omits the gates between the boxes: who calls a thing done, what a finding must carry, who may close it, and who reads each escape.

The seven stages name work the team already does

Requirements review, test planning, test case design, execution, defect reporting, regression, release: that is the list, give or take a box. Read it against a five-person team shipping weekly and every box is already occupied. Somebody reads the ticket before starting. Somebody clicks through the feature before merging. Somebody files what they find, somebody checks the old paths, somebody presses deploy. Naming those activities changes what they are called and nothing about what they let through.

The bugs that reach customers do not come from a missing stage. They come from the joints: the story that was called done because the engineer was finished rather than because anybody checked it against the criteria; the finding filed as “checkout is broken” that nobody could reproduce and so nobody fixed; the ticket closed by the person who wrote the fix. Each of those is a decision that got made by default, and a stage diagram has no place to write a decision down.

A checklist that lists activities is a schedule. The process is the set of rules about what has to be true before the next activity is allowed to start.

A gate is a question with a named owner

Four gates cover most of what a small team lets through. Each one sits between two stages, asks one question, and names who is allowed to answer it.

BetweenThe gate asksWho answersWhat passes without it
Building and calling it doneDoes the behaviour match every line of the definition of done?Somebody other than the authorWork that is finished from the inside and unchecked from the outside
Finding and filingCan a person who was not there reproduce this from the ticket alone?The reporter, against a written minimumDescriptions of defects that close as irreproducible
Fixing and closingHas the reporter, or a test written from the report, confirmed the fix?The reporter or the tester, never the fixerFixes that address the symptom the engineer imagined
Releasing and forgettingWhich gate did each escaped defect pass through, and why?The lead, per escapeThe same gap, next release

The third column is the one the diagrams leave out, and it is where the process lives. A question with no owner gets answered by whoever is in a hurry, and the person in a hurry is the author. A definition of done that the engineer checks against their own work is a note to self.

Assurance and control are two jobs held by one person

The distinction between quality assurance and quality control matters here, and it is usually presented as a matter of vocabulary. Assurance is the design of the gates: what the questions are, who owns them, what a finding must carry. Control is walking through them on a Tuesday afternoon with a build in front of you.

In a large organisation different people do those jobs, and the friction between them is what keeps the gates honest. In a team of six the person who wrote the gate is the person passing through it, and the danger is specific: the gate drifts to whatever that person feels like checking today. The defence is the same one every small team uses for everything else. Write it down. A gate that exists as a sentence in a document can be pointed at when it is skipped, and a gate that exists as a habit cannot.

That also settles what the lead’s own job is. Somebody told to put a QA process in place is being asked to do assurance: to write the gates, not to run every check. Running the checks is what the gates then distribute to whoever stands on the other side of them.

Measure the process by what escapes it

A process is not proven by the count of bugs it catches. A team that finds many bugs before release might be catching a lot or might be writing a lot, and the number alone cannot say which. The count that describes the process is the escapes: defects that reached a customer after release. Divide the defects found before release by that number plus the escapes and the result is defect removal efficiency, the share of a release’s defects the process removed before shipping. It needs no tooling beyond two columns, a release date, and an agreed window after it in which production is allowed to reveal what testing missed.

What makes it useful is the question each escape then gets asked: which gate did it pass through? A defect that shipped because the story was called done without a check against the criteria is a failure of the first gate. One that was filed, closed as irreproducible and later hit by a customer is a failure of the second. Over a quarter the escapes cluster, and the cluster says which gate to tighten, which is a more specific instruction than “test more”.

The evidence gate is the one a small team skips

The second gate, the one between finding and filing, is where a small team most often has nothing written. A tester, a colleague in support or a client reports that the export is wrong. The engineer opens the export and it is fine. The finding closes, and it is an escape waiting to happen.

The minimum that gate asks for is the expected result, the observed result, and the conditions: browser, viewport, route, the console and network activity in the seconds before. Most of that is out of reach for the person reporting, so a gate that asks them for it will be waived. A report filed from the page itself carries those conditions automatically, which is what evidence capture does: the ticket arrives with the environment and the failed request attached, and reproduction starts from the ticket rather than from a reply asking which browser it was.

The count of escapes is a floor

The escape count only records defects a customer reported, and no gate changes that. A customer who hit a broken export, sighed and went elsewhere is an escaped defect nobody counted, and no gate in the process sees them. The escape count is therefore a floor and the efficiency figure a ceiling: the process is at least this leaky, and an improving figure can mean tighter gates or quieter customers.

A channel for reports therefore has to exist on the product itself, close to where the failure happened, because a report that costs a customer an email will mostly not be sent. The lead, for the same reason, reads every escape as a sample and treats the gate it passed through as the gate that let others through unseen. That is the whole process: four questions with owners, and a habit of asking which one failed.

Common questions

What are the stages of a QA process?

Most guides list seven: requirements review, test planning, test case design, test execution, defect reporting, regression testing and release. A small team already performs all of them informally. The stages describe activities; what decides whether defects reach customers is the gates between them, meaning the questions that have to be answered before the next activity starts.

How do you set up a QA process for a small team?

Write down the gates rather than the stages: what a story has to satisfy before it is called done, what a finding has to carry before it is filed, who confirms a fix before it is closed, and who reviews each defect that reaches a customer. Each gate needs a named owner who is not the author of the work.

What is a gate in a QA process?

A gate is a question that has to be answered before the next activity is allowed to start, with a named owner who is not the author of the work: does the behaviour match the definition of done, can a stranger reproduce the finding from the ticket, has the reporter confirmed the fix, which gate did this escape pass through. Stages describe activity; gates decide what passes.

How do you measure whether a QA process is working?

Count the defects found before release and the defects that reached customers in an agreed window after it, and divide the first by the sum of both: that share is defect removal efficiency. Then ask of each escape which gate it passed through. The escape count is a floor, because it only includes defects somebody reported, so the efficiency figure is a ceiling, and the gate the escapes cluster around is the one to tighten.

Does a QA process need a dedicated QA engineer?

A dedicated tester makes the gates easier to hold, because the person checking is never the person who built. Without one, the process still works if the gate's owner is anybody other than the author: a second engineer, a support colleague, or the reporter confirming the fix.

The Notarus team

The people who build Notarus. We write up the decisions behind the product when the reasoning is worth more than the changelog line.

Keep reading

Reports with the evidence attached.

Start free, one script tag.

<script src="…/widget.js" data-key="pb_pub_…">