QA in agile happens at three sprint boundaries

When a team goes agile the test phase goes and nothing replaces it. QA in agile happens at three boundaries in the sprint, and the tester owns each one.

Three offset glass rings encircle a core, with the middle boundary glowing orange

The team went agile, the test phase went with it, and nothing was put in its place. Testing is everybody’s job now, which in practice means it is nobody’s. Testing on an agile team happens at three boundaries in the sprint rather than in a phase, and the tester’s job is to own each of them.

The phase went away and nothing took its place

Under the old process testing had a slot. Development finished, a build was handed over, a tester worked through it, and a report went back. The slot was late and the report often arrived after the decision it was meant to inform, which is a fair part of why the team changed process. What the change did not supply was a replacement for the slot.

The Scrum Guide names no tester. It names one product owner, one scrum master and the developers, says the team is cross-functional with every skill it needs, and says work that does not meet the definition of done is not part of the increment. Testing skill is expected to be inside the team; where it sits is left to the team. So a tester on a newly agile team is handed one of two arrangements. Either testing gets squeezed into the last days of the sprint, which is the old phase in a smaller box and with less notice, or the team announces that everybody tests, which is a sentence about responsibility that names nobody.

Both arrangements have the same symptom. Findings arrive at the end, when there is no time to act on them, and the tester is asked why the sprint was not tested sooner.

Testing moved to three boundaries in the sprint

The phase mattered less than what it guaranteed, which was that somebody with a tester’s habits looked at the work before it reached a user. A sprint has three moments where work changes state, and each one is where that look belongs.

BoundaryWhat crosses itWho does the workWhat the tester owns
A story enters the sprintA story with acceptance criteriaThe product owner writes the intent, the developers estimate itThat the criteria are testable: written as examples, with the failure paths named
A story is called doneCode with its own testsThe developer writes the unit and integration tests and runs them on every pushAn exploratory session against the story on the preview, with a charter and a timebox
The increment shipsA build for usersThe whole teamThe regression call: what the risk-based pass covered, what it skipped, and whether to ship

The first boundary is where the leverage moved. An acceptance criterion written as an example, a cart with one item in it and the empty state that removing it shows, is a test that exists before any code does. A developer builds to it, and the ambiguity that would have been found in the old test phase is found in refinement, when fixing it costs a conversation.

The second is where the developer’s tests stop and the tester’s session starts. Tests a developer writes prove the code does what the developer understood. The session finds where the developer’s understanding and the requirement parted, and a session-based pass with a one-line charter and a fixed timebox finds it inside the sprint rather than after it.

The third is the one no other role can hold, and the last section is about it.

What everybody tests has to mean

The sentence is right as long as it is about evidence rather than judgement. Everybody tests means everybody produces a record of what they checked. A developer’s record is the automated tests for the code they wrote, running on every push. The product owner’s record is an acceptance against the criteria, story by story. The tester’s record is the session notes and the regression call.

What the sentence cannot mean is that the judgement about what was left untested belongs to everyone, because a judgement held by everyone is made by no one. Somebody has to be able to say, at the sprint review, that the payment flow was walked, the export was not, and the team is shipping with that exposure. When that sentence has no owner it does not get said, and the exposure is discovered by a customer.

So the tester’s title may go. The accountability stays with one person, whatever the team calls them.

Where a finding goes when there is no test phase

In the old process findings went into a report at the end. Without a phase they arrive throughout the sprint, and they compete with the sprint’s stories for the same hours. The rule that keeps that from becoming a fight is short. A finding on a story still in the sprint goes back to that story, because the story is not done. A finding on behaviour that already shipped is a regression, and it goes to triage on the board like any other report, with its severity and its priority set as two separate fields, so that it earns sprint time on its merits rather than on its noise.

What makes the rule workable is the quality of the finding on arrival. A developer mid-story cannot afford a conversation about which browser it was. So the tester files from the page under test, and the ticket opens with the console errors, the failed requests, the viewport and the page path that a developer would otherwise have to ask for; pushed on to the team’s tracker, it keeps its stage in step in both places, so nobody re-keys it. A finding that arrives that way is actionable at the next stand-up; one written from memory in a tracker form is a question, and questions wait.

Regression testing in agile covers the other half, which is how the regression pass itself gets chosen when a sprint has no room for all of it.

The regression call belongs to one person

A sprint has no room for a full regression pass, and pretending otherwise is how the pass gets skipped in silence. The pass is risk-based: the areas this sprint touched, the areas that have broken before, the areas whose failure costs most. Everything else is untested by decision, and the decision has to be written down and accepted at the review by the product owner, out loud, so that the exposure is chosen rather than discovered.

That naming is the one part of the job that cannot be automated and cannot be shared. A suite reports what it ran. It cannot report what nobody thought to run, and a team of developers each confident in their own tests has no view of the whole. The tester’s view of the whole is what the team lost when it lost the phase, and it comes back only if one person is asked for it at every sprint end.

The sprint’s shape adds a second limit. The last stories called done in a sprint get no exploratory session, because the sprint ends before anyone can run one. Either the session is inside the story’s estimate, so a story is not done until it has been walked, or the end of every sprint is the least-tested part of it by construction. A team that has not decided has picked the second, and the definition of done is where the choice gets made explicit.

Common questions

When does testing happen in an agile sprint?

Testing in an agile sprint happens at three boundaries rather than in a phase: when a story enters the sprint, where the acceptance criteria are made testable; when a story is called done, where somebody other than its developer walks it on the preview; and when the increment ships, where a risk-based regression pass is run and what it skipped is named.

What is the role of QA in an agile team?

The role of QA in an agile team is to own the three boundaries where testing happens. A tester writes acceptance criteria as examples a developer can build to, runs exploratory sessions against stories that meet their definition of done, and makes the regression call at the end of the sprint, naming what the pass covered and what it did not.

Does everyone test in agile, or is there still a tester?

Everyone produces evidence of what they checked: developers through the tests they write for their code, the product owner by accepting against the criteria. The judgement about what was left untested and whether to ship anyway is one job, and spreading it across the team is how it stops being done.

Where do bugs found mid-sprint go?

A finding on a story still in the sprint goes back to that story, which is not done. A finding on behaviour that already shipped is a regression and is triaged on the board like any other report, with a severity and a priority set separately, so it competes for sprint time on its merits.

Why are the last stories in a sprint the least tested?

The last stories called done in a sprint get no exploratory session, because the sprint ends before anyone can run one. Unless the session sits inside the story's estimate, so that a story is not done until somebody other than its developer has walked it, the end of every sprint is its least-tested part by construction. The definition of done is where a team makes that choice explicit.

The Notarus team

The people who build Notarus. We write up the decisions behind the product when the reasoning is worth more than the changelog line.

Keep reading

Reports with the evidence attached.

Start free, one script tag.

<script src="…/widget.js" data-key="pb_pub_…">