Test execution
Test execution is the phase of the testing process in which the tests that were planned and designed are run against the build under test, either by a person following the test cases or by an automation tool, and the outcome of each is recorded as passed, failed, blocked or skipped, with each failure analysed and a defect report filed when the product is at fault.
The longer version
Execution begins when the entry criteria are met: the build is deployed to the test environment, the test data is loaded, and the test cases are approved. A tester takes a test case, performs each step, compares the actual result with the expected one, and marks the outcome. A failure is checked against the test case and the environment, and gets a defect report with steps and evidence when the product is at fault. A test that cannot be run because something it depends on is broken is marked blocked, not failed, and the blocker is itself reported. Automated suites do the same with a runner and a results file, and someone still has to read the failures.
Test execution is separate from test design, which happens before it and produces the cases, and from test closure, which happens after it and judges the results. Teams that blur design into execution write the test as they run it, which is exploratory testing and should be reported as such. The other confusion is between execution and the test report: execution produces the raw log of outcomes, one row per test per run; the report is the summary somebody writes from that log for people who will not read it.
Execution stalls on things that are not tests. The environment is down, or a version behind, or shared with another team's data. The test data was reset overnight. The build under test changed halfway through the cycle, so half the results describe a build that no longer exists. Each of those is recorded as blocked, and a cycle with more blocked than failed is a finding about the environment rather than the product. The other drain is defects logged without evidence, which bounce back from developers and consume the tester's next execution slot in reproducing yesterday's finding.
Common questions
What are the entry criteria for test execution?
The build is deployed to the agreed test environment and smoke tested, the test data is in place, the test cases for the cycle are reviewed and approved, and the defect tracker is ready to receive reports. Starting execution without these produces blocked results that look like testing but measure nothing.
What is the difference between a failed test and a blocked test?
A failed test ran to the point of comparison and the actual result did not match the expected one, which is a defect in the product. A blocked test could not reach that point because something it depended on was unavailable: a broken environment, missing data, or an earlier failure. Counting blocked tests as failures misstates the product's quality.
Who performs test execution?
Testers on the QA team for scripted manual cases, an automation runner for automated suites, and often developers for unit and integration levels. In user acceptance testing the customer's representatives execute. Whoever runs a test owns the accuracy of its recorded outcome and the evidence attached to any failure.
How long does test execution take?
As long as the number of test cases, the setup and run time of each, and the rate of failures that need investigating add up to, which the test plan estimates from previous cycles. The estimate is usually wrong in one direction: blocked environments, unstable builds and defects filed without evidence stretch execution past the plan, while the tests themselves rarely take longer than expected.
What is a test cycle?
A test cycle is one complete pass of a set of test cases against one build in one environment, with every outcome recorded. A release usually has several: an initial cycle on the first candidate build, then a regression cycle on each build that fixes defects, until the exit criteria are met. Results are reported per cycle so that a pass recorded against an earlier build is never mistaken for a pass on the one shipping.
Bug reports that arrive ready to triage.
Collect replay, console, network and browser context with the Free plan.