A regression reaches production despite green unit tests, and the development team immediately considers an end-to-end test for every feature. Far fewer are needed: one per critical journey.
An end-to-end test (abbreviated E2E) runs a complete user journey on the deployed application: it goes through the interface, the services, the database and the neighbouring systems, as a person would. Among automated tests, it alone checks that the assembled whole works. It is also the slowest test, the most costly to maintain and the most exposed to failures with no real cause. It is therefore reserved for journeys whose failure would halt the business. The method follows an order: choose these journeys, prepare the data and environment, stabilise the suite, then give it its place in the delivery pipeline.
The end-to-end test at the top of the pyramid: rare because expensive
The test pyramid, presented in our page software quality and testing, places the end-to-end test at its top, in small numbers. This position stems from three costs.
The first is execution. The test starts a browser, loads pages and waits for network responses: its duration is counted in seconds or minutes, whereas a unit test returns its verdict almost immediately. The second is diagnosis. A failure designates a journey, without saying which of the components traversed is at fault. The third is maintenance. A modified screen, missing data or an unavailable neighbouring service is enough to make a test fail whose subject has not changed: this is the source of its instability.
In return, it detects what no lower level sees, assembly defects: an environment-specific configuration, a wrongly declared route, authentication that does not propagate from one service to another, a database migration not applied. Each component passes its own tests, and the journey fails.
A journey can also be replayed through the programming interfaces alone, without a browser. The test remains end-to-end for the services, it gains in speed and stability, but it no longer checks the screen.
Choosing the journeys that deserve an end-to-end test
A journey goes on the list when it combines several of these characteristics:
- its failure interrupts a sale, a payment, a subscription, the submission of an application or access to the service itself;
- it goes through several systems, at least one of which the team does not control;
- a large share of users take it;
- no lower-level test can provide proof of it.
The splitting rule counts as much as the selection: one test per nominal journey. “Ordering with a valid card” justifies an end-to-end test. The many reasons for a card being declined are checked in the unit and integration tests of the payment service, where each one runs quickly and fails for a readable reason.
This step produces a short list of journeys, named in business vocabulary. The product owner answers for their criticality, the development team for their feasibility. Any later addition goes through this list.
Preparing the data and the environment
Each test creates the data it needs (an account, a basket, a contract) through an API call or a script, without going through the interface, and assumes nothing about the state left by another. Two tests that share the same customer account are likely to fail as soon as they run in parallel. Data sets are built from fictitious data. A production copy containing personal data raises the questions covered in our article on data anonymisation.
The environment receives the application in the version that will go to production, with a configuration as close as possible. Third-party services (payment, messaging, identity verification) are called in their provider’s sandbox where one exists, or replaced by a simulator. In the second case, the test no longer proves the real connection, and this limit is written down.
An environment created for each run, then destroyed, removes residual states. Providing it falls under the platform described in our page DevOps and platform engineering.
At the end of this step, the pipeline can create and reset, without intervention, a reproducible data set and a complete environment. The development team owns the data, the platform team the environment.
Stabilising the suite by addressing each intermittent failure at its cause
An unstable test (flaky test) fails then passes without the code having changed. Four causes come up regularly.
The first is the fixed-duration wait: a pause of a few seconds is too short on a slow day and useless the rest of the time. The test should instead wait for an observable state, such as a button that has become active or a message displayed. Tools such as Playwright or Cypress include this mechanism: they check that an element is ready before acting and retry up to a time limit.
Next come selectors tied to the layout. A test that designates an element by its position on the screen breaks at the first change of mock-up. The element’s role, its label or an attribute reserved for tests are more resilient.
The last two causes are dependency between tests, through execution order or shared data, and the environment itself: a slow third-party service, a background job, the clock.
Automatic rerunning of a failed test, offered notably by Playwright and Cypress, serves to distinguish an outright failure from an intermittent one. It fixes nothing: a test that passes only on the second attempt is counted as unstable and assigned to someone. If it is not fixed quickly, it leaves the blocking path and goes into quarantine, with an owner and a deadline, recorded in the same backlog as the rest of technical debt. Once the deadline has passed, it is repaired or deleted.
Screenshots, video recordings and the trace of network exchanges, kept at each failure, shorten diagnosis. The test code goes through the same code review as the application code. The step delivers a suite in which a failure signals a defect; the team developing the journey answers for it.
Placing the end-to-end test in the delivery pipeline
| Moment | What runs | Effect of a failure |
|---|---|---|
| On each merge request | The few vital journeys, on an environment created for the occasion | The merge is blocked |
| After deployment to the integration environment | The full suite, spread over several parallel runs | Promotion to production is blocked |
| After the production release | A few journeys with no side effects, replayed at regular intervals | An alert is raised |
Duration drives this distribution. A suite longer than the team is willing to wait for gets bypassed: it is parallelised, or it slims down. Blocking status must be earned too. It is granted only to stabilised tests, otherwise the pipeline stops for false alarms.
Journeys replayed in production use reserved accounts and write nothing that commits the organisation: no real order, no message to a customer.
This set-up checks that a journey works for a user. Whether the same journey holds up under high traffic falls under load testing, which follows other rules. The team in charge of the delivery pipeline answers for execution, the development team for the verdict.
A short suite is defended at every incident
Each defect that escapes to production suggests adding an end-to-end test. Before writing it, the team asks itself at which level the defect could have been detected. If it concerned a business rule or a contract between two services, the test is added at that level.
The suite is then reread at each major evolution of the product: a journey that has ceased to be critical loses its test, a journey that has become critical receives one, and the list validated by the product owner remains the reference.