Risk-based testing

Risk-based testing is a strategy that decides what to test, how deeply and in what order by ranking the product's features against the likelihood that each will fail and the damage a failure would do, so that limited testing time goes to the areas where a defect would cost the most. It is the alternative to trying to test everything to the same depth.

The longer version

The method starts with a list of features or components and two scores for each: how likely it is to contain a defect, judged from complexity, recent change, past bugs and the team's confidence; and how much harm a defect would do, judged from revenue, safety, data and reputation. The product of the two ranks the list. The top of the list gets the most test design, the most execution and the earliest slot; the bottom gets a smoke test or nothing, with that decision written down. When time runs out, what is left untested is known and was chosen.

The contrast is with coverage-driven testing, which tries to test every requirement to the same depth, and with schedule-driven testing, which tests whatever fits before the date. Risk-based testing is often confused with severity and priority on a bug, but those are properties of a defect already found; risk is a property of an area where no defect has been found yet. The scores are also distinct from business value: a highly valued feature that is stable and unchanged can rank low, and an obscure settings page that touches billing can rank high.

Risk scores go stale. A ranking made at the start of a project and never revisited misses the module that was rewritten last month, and the team tests the areas that were risky a year ago. The second failure is scoring by whoever fills in the spreadsheet rather than by the people who know: developers know where the fragile code is, support knows where customers get hurt, and a score set without both is a guess. The third is treating the low-risk tail as safe rather than as untested; the test report should say which areas were deliberately skipped.

Common questions

How is risk calculated in risk-based testing?

Each feature or component gets two scores: the likelihood of a defect, judged from complexity, amount of recent change, defect history and how well the team understands it; and the impact of a defect, judged from revenue, data, safety and reputation. Their product ranks the list. The scale is coarse on purpose, since the point is ordering, not precision.

Who decides the risk scores?

The scoring is a joint decision, usually led by the test lead, with developers contributing likelihood because they know where the code is fragile, and product or support contributing impact because they know what a failure costs. A score set by one person alone is a guess dressed as a method.

What is the difference between risk-based testing and priority on a bug?

Priority on a bug ranks a defect that has already been found against other work. Risk in risk-based testing ranks an area of the product before testing, by how likely it is to hide a defect and how much that defect would cost. One orders the fix queue; the other orders the test plan.

When should risk-based testing be used?

Whenever the testing needed to cover everything to the same depth exceeds the time available, which is nearly always. Risk-based testing is most valuable on large systems with uneven risk, on releases with a fixed date, and on regression suites that have grown past what a cycle can run. On a small product with a short suite the ranking adds little, because everything gets run anyway.

How does risk-based testing work in an agile team?

The ranking happens per iteration rather than per project. When a story is refined, the team scores its likelihood of failure and the impact of that failure, and the score decides how much test design the story gets, whether it needs exploratory sessions on top of its acceptance tests, and whether its area joins the regression run. The scores are revisited as the code changes, which a project-level risk register never manages.

Bug reports that arrive ready to triage.

Collect replay, console, network and browser context with the Free plan.

<script src="…/widget.js" data-key="pb_pub_…">