Test automation
Test automation is the practice of writing code that exercises software and checks the result without a person present, so that the same tests run identically on every build. An automated test drives the product through its interface or its code, compares the outcome with an expected value, and reports pass or fail. The practice covers writing, running and maintaining those tests, not the tools that run them.
The longer version
An automated test has three parts: setup that puts the product in a known state, an action that exercises it, and an assertion that compares the outcome with what was expected. Tests are organised by how much of the system they touch. Unit tests call one function in isolation. Integration tests check that components work together against real dependencies. End-to-end tests drive the product through its user interface as a person would. The higher the level, the more the test proves and the slower and more brittle it is, which is why suites are usually shaped with many small tests and few large ones.
Automation is contrasted with manual testing, and the mistake is to see it as a replacement rather than a division of labour. Automation owns repetition: regression checks that must run on every commit, identically, which people do badly and resent. Manual testing owns judgement. Automation is also confused with continuous integration, which is the system that runs the tests on every change; a team can have a CI pipeline that runs no meaningful tests, and an automated suite that somebody runs by hand on a Friday. The suite is the tests; CI is the scheduler.
Automated suites decay. Every test is code that has to change when the product changes, and a suite nobody maintains fills with tests that are skipped, tests that pass for the wrong reason, and tests that fail intermittently until the team stops reading the results. A flaky suite is worse than a small honest one, because a red build that is usually a false alarm trains everyone to ignore the real one. The other failure is automating the wrong layer: driving every check through the user interface, where tests are slowest and break on every layout change, when most of them could run against the code underneath. What automation cannot do is notice the thing nobody wrote an assertion for.
Common questions
Who is responsible for test automation?
The people who change the code, in the main. Unit and integration tests are written by the developers whose code they cover, because a test somebody else wrote breaks the moment the code moves. End-to-end suites are often owned by a test engineer or a QA team, and that split is where suites decay, since the owner of the test is not the owner of the change that broke it.
What should be automated first?
The checks that run most often and change least: the smoke test that proves the build starts and the core flows work, then the regression cases that have caught real bugs before. Automate at the lowest level that proves the point, and leave interface-driven tests for the few flows where the interface is what is being tested.
Is test automation the same as continuous integration?
Continuous integration is the system that builds the product and runs checks on every change. Test automation is the set of scripted tests it can run. One is the scheduler and the other is the content, and each is possible without the other: a pipeline that runs no real tests, or a suite that somebody triggers by hand.
Why do automated tests become flaky?
Because something the test depends on is not under its control: a timeout set for a fast machine, an order of asynchronous responses that varies, shared state left behind by another test, a third-party service that answers differently, a date that crosses midnight during the run. Each cause is fixable once found, and a flaky test left in the suite trains the team to rerun red builds until they pass.
Does test automation find new bugs?
Rarely, and by design. An automated test asserts an expected result somebody already knew, so it finds a bug the first time it runs against code that violates it, and after that it finds regressions: the same check failing on a later build. New bugs in new behaviour are found by the person writing the first test for it or by manual and exploratory testing, and automation then keeps them from returning.
Bug reports that arrive ready to triage.
Collect replay, console, network and browser context with the Free plan.