Characterization testing
Characterization testing is the writing of tests that capture what existing code currently does, without judging whether that behaviour is correct, so that a later change can be checked against it. It is used on legacy code with no tests before refactoring, and it is also called golden master testing or approval testing when the captured behaviour is a stored output the new output is compared against.
The longer version
The procedure inverts normal test writing. Instead of deciding the expected result and writing a test that asserts it, the tester runs the existing code with a chosen input, records whatever came out, and writes a test asserting that output. For a function that formats an invoice, the test says the invoice looks exactly like this, including the misaligned column nobody has fixed. Golden master testing does the same at larger scale: run the whole report generator over a fixed set of inputs, store every output as the master, and diff future runs against it.
Characterization tests differ from regression tests in what the assertion means. A regression test asserts the behaviour the specification requires; a characterization test asserts the behaviour that happens to exist, bugs included. The difference matters when a test fails: a regression test failing means the code is wrong, a characterization test failing means the code changed, and a person has to decide whether the change was the refactoring going wrong or a bug being fixed. The tests are a safety net for change, not a statement of correctness.
The recorded output was never verified, and teams forget that. A team refactors a pricing module, keeps every characterization test green, and ships a rounding error that was there all along, now with a test that protects it. The second failure is a golden master so large that every intentional change diffs hundreds of lines, so the team starts approving diffs without reading them. Keeping the inputs deliberate, and retiring characterization tests in favour of specified ones as understanding grows, is what keeps the technique honest.
Common questions
What is the difference between characterization testing and regression testing?
Regression testing asserts the behaviour the requirements specify and fails when the code is wrong. Characterization testing asserts whatever the code did when the test was written, correct or not, and fails when the code changes. Characterization tests are written on code with no specification to test against; regression tests are written from one.
When should you write characterization tests?
Before changing code that has no tests and no reliable specification, usually legacy code that nobody fully understands. The tests pin the current behaviour so the change can be made with confidence that nothing observable moved. Once the behaviour is understood, they are replaced with tests that assert what the code should do.
Is golden master testing the same as characterization testing?
Golden master testing is characterization testing at output scale: the whole output of a run is stored and future runs are compared against it, rather than individual assertions being written. Approval testing is the tooling name for the same mechanism: an output is approved once and later runs are compared to it. All three record existing behaviour rather than specified behaviour.
How do you write a characterization test?
Pick an input the code is likely to see in production, run the code, and capture the output exactly, including whatever looks wrong. Write a test that asserts that output. Run it to confirm it passes, then change the code in a way that should alter the output and confirm the test fails, which proves it is watching the right thing. Repeat with inputs that reach different branches until the behaviour about to be changed is pinned.
Do characterization tests replace unit tests?
Characterization tests are a stopgap rather than a replacement. They exist because the code has no unit tests and no reliable specification, and they hold the behaviour still while it is refactored. Once the refactoring has made the code understandable, the team writes unit tests that assert what the code should do and retires the characterization tests that assert only what it did. A suite that keeps both forever protects bugs alongside features.
Bug reports that arrive ready to triage.
Collect replay, console, network and browser context with the Free plan.