Exhaustive testing
Exhaustive testing is a test approach in which every combination of input values and preconditions is tested. For anything larger than a trivial program the number of combinations is too large to run, so exhaustive testing is treated as impossible in practice, and testers run a small set of tests chosen by risk and technique.
The longer version
A small order form shows the scale. It has a quantity field that accepts 1 to 99, a size list with 5 options, a colour list with 8, a gift-wrap tick box and 3 delivery choices. The valid combinations alone number 99 × 5 × 8 × 2 × 3 = 23,760. At one minute each, running them by hand takes 396 hours. Add an optional gift message and the count moves out of reach. A message of exactly five lower-case letters has 26 × 26 × 26 × 26 × 26 = 11,881,376 possible values, and 23,760 × 11,881,376 comes to 282,301,493,760 combinations.
That figure still leaves most of the input space out. It ignores invalid entries such as a quantity of zero or a letter in the quantity field, messages of any other length, and every precondition: whether the customer is signed in, what is in stock, which browser is in use, what was in the basket before. The ISTQB glossary's definition of exhaustive testing takes in all combinations of input values and preconditions. The ISTQB Foundation syllabus lists "Exhaustive testing is impossible" as the second of its seven testing principles and states that testing everything is not feasible except in trivial cases.
The same principle says what to do in its place: use test techniques, test case prioritisation and risk-based testing to focus the effort. Equivalence partitioning cuts the quantity field from 99 values to a few classes, and boundary value analysis adds the edges at 0, 1, 99 and 100. Pairwise testing covers every pair of options across the lists with far fewer cases than the full set. Risk-based testing decides which areas deserve the deepest design, and prioritisation decides which tests run first when time is short. None of this proves that the untested combinations work. It makes the selection deliberate, so the team can say what was left out and why.
Common questions
What is exhaustive testing?
Exhaustive testing is testing every possible combination of inputs and preconditions for a piece of software. The ISTQB glossary defines it as a test approach in which the test suite comprises all combinations of input values and preconditions, and gives complete testing as a synonym.
Why is exhaustive testing impossible?
The number of combinations grows by multiplication with every field, option and precondition, and a single free-text field allows more values than anyone could run. A form with a handful of inputs can have billions of combinations, so no team has the time to execute them all or to check the results.
Is exhaustive testing ever possible?
Exhaustive testing is possible only where the inputs and preconditions are few enough to list, the exception the ISTQB principle itself allows. A function with two on-off inputs has 2 × 2 = 4 combinations and can be tested completely. The same holds for any component whose inputs and preconditions are few and can be listed in full.
What do testers do instead of exhaustive testing?
Testers choose a subset of tests on purpose. Risk analysis shows where a failure would cost most, equivalence partitioning and boundary value analysis reduce each input to a few representative values, pairwise testing thins the combinations, and prioritisation orders what remains so the most important tests run first.
Which ISTQB testing principle covers exhaustive testing?
Exhaustive testing is the subject of the second of the seven testing principles in the ISTQB Foundation syllabus, titled "Exhaustive testing is impossible". The principle sets trivial cases aside and turns the tester's job into selection: choosing tests by technique, by priority and by risk.
Bug reports that arrive ready to triage.
Collect replay, console, network and browser context with the Free plan.