Test case prioritization
Test case prioritization is ordering a test suite so the tests most likely to fail on the current change run first. The whole suite still runs, but the first minutes of it carry the highest chance of finding a defect, so a developer learns about a break early and a run can be stopped at the first failure.
The longer version
An ordering needs a score per test, and the score comes from signals the team already has. Tests that failed in recent runs rise. Tests covering the files changed in this commit rise. Tests tied to features with a history of defects, or to high-risk areas such as payments and sign-in, rise. Tests that never fail and take a long time sink. The scores are sorted, the suite runs from the top, and the pipeline reports the first failure as soon as it appears rather than at the end.
Test impact analysis shortens the run by dropping tests a change cannot affect, while prioritization drops nothing and only reorders. Risk-based testing is the planning-level cousin: it decides which areas deserve test effort at all, where prioritization decides the running order of tests that already exist. A suite can be reordered by a rule as simple as running last night's failures first, which is crude and effective, and needs no coverage map.
The weak point is the signal going stale. An ordering built from last quarter's failures puts the tests for a retired feature first and the newly risky area last. Flaky tests distort it: a test that fails at random looks like the most valuable one in the suite and floats to the top, where its noise delays the real signal. And a suite ordered for quick failure feedback can leave the slowest tests untouched for weeks if the run is routinely cut short after the first failure.
In a pipeline, prioritization applies on every commit and costs almost nothing, since it changes order and not content. Teams pair it with a full run before release, so a test deprioritised for a month still gets executed before the build ships.
Common questions
What is the difference between test case prioritization and test selection?
Prioritization orders the complete suite so likely failures surface first, and every test still runs. Selection, which test impact analysis performs, picks a subset for the change and skips the rest. Selection saves run time; prioritization saves time to first failure.
What criteria are used to prioritise test cases?
Recent failure history, coverage of the files the change touched, the business risk of the feature, the defect history of the area and the run time of the test. Teams weight these differently, and a simple rule such as running last run's failures first already captures most of the benefit.
Does test case prioritization reduce the number of tests that run?
Prioritization reorders and reduces nothing by itself. The saving comes from stopping early once a failure is found, or from giving developers the first results while the rest of the suite is still running. Reducing the count is the job of selection, which is a different technique.
How do flaky tests affect test case prioritization?
A flaky test fails intermittently, so a failure-history score ranks it among the most valuable tests and runs it first. Its noise then delays or hides the genuine signal. Quarantine flaky tests, or exclude their failures from the score, before relying on a history-based ordering.
How often should the test order be recalculated?
On every run if it is derived from the change and recent failures, since the inputs change with each commit. A fixed ranking set by hand, such as critical paths first, should be reviewed when the product's risks shift, because a ranking from last year orders for features that may no longer matter.
Bug reports that arrive ready to triage.
Collect replay, console, network and browser context with the Free plan.